-- Logs begin at Thu 2022-07-14 10:50:40 UTC, end at Thu 2022-07-14 12:05:10 UTC. -- Jul 14 11:22:31.061937 dv-cinder-849803-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 11:22:31.074708 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 11:22:31 2022] *** Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: nodename: dv-cinder-849803-1-iscsi Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: machine: x86_64 Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: clock source: unix Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: pcre jit disabled Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: detected number of CPU cores: 4 Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: current working directory: / Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: detected binary path: /usr/bin/uwsgi-core Jul 14 11:22:31.075995 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: your processes number limit is 47680 Jul 14 11:22:31.076707 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: your memory page size is 4096 bytes Jul 14 11:22:31.076707 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: detected max file descriptor number: 2048 Jul 14 11:22:31.076707 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: lock engine: pthread robust mutexes Jul 14 11:22:31.076707 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: thunder lock: enabled Jul 14 11:22:31.076707 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 11:22:31.076707 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 14 11:22:31.121644 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: Python main interpreter initialized at 0x561c4475b9c0 Jul 14 11:22:31.121644 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: python threads support enabled Jul 14 11:22:31.121644 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: your server socket listen backlog is limited to 100 connections Jul 14 11:22:31.121644 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: your mercy for graceful operations on workers is 90 seconds Jul 14 11:22:31.121979 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 11:22:31.122055 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: *** Operational MODE: preforking *** Jul 14 11:22:31.122320 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: *** uWSGI is running in multiple interpreter mode *** Jul 14 11:22:31.122320 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: spawned uWSGI master process (pid: 80421) Jul 14 11:22:31.122517 dv-cinder-849803-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 11:22:31.124257 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: spawned uWSGI worker 1 (pid: 80433, cores: 1) Jul 14 11:22:31.124257 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: spawned uWSGI worker 2 (pid: 80434, cores: 1) Jul 14 11:22:31.124257 dv-cinder-849803-1-iscsi devstack@keystone.service[80421]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 11:22:36.744334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 11:22:36.744334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: from cryptography.utils import int_from_bytes Jul 14 11:22:36.745117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 11:22:36.745117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: from cryptography.utils import int_from_bytes Jul 14 11:22:36.754695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 11:22:36.754695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: from cryptography.utils import int_from_bytes Jul 14 11:22:36.755194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 11:22:36.755194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: from cryptography.utils import int_from_bytes Jul 14 11:22:36.800568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 11:22:36.800568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 11:22:36.800568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] command line args: [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 11:22:36.800823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 11:22:36.801042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ================================================================================ {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 11:22:36.801291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] admin_token = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.801550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] config_dir = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.801735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.801843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 11:22:36.801934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] config_source = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.802015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 11:22:36.802095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] debug = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.802214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] command line args: [] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 11:22:36.802299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.802522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 11:22:36.802643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.802720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ================================================================================ {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 11:22:36.802821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.802899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] admin_token = **** {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.803110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.803196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] config_dir = [] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.803314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.803391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.803500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] config_source = [] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.803581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.803657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] debug = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.803764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_config_append = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.803869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.803965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.804092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_dir = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.804181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.804291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.804372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_file = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.804505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.804606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.804765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.804858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.804983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] list_limit = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.805088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.805174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] log_config_append = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.805251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.805380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.805469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.805532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] log_dir = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.805589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.805737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] log_file = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.805863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.805960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.806074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.806181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.806339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.806446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.806581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.806746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.806819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.806877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.807022 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.807108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.807262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.807372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.807509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.807628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.807746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.807848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.807972 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.808184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.808301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.808420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.808544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] publish_errors = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.808657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.808767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.808884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.809026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.809226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.809396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.809519 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.809624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.809732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.809850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] publish_errors = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.809972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] standard_threads = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.810086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.810283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.810401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.810520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.810640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.810759 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.810759 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] use_journal = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.810983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.811096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] use_json = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.811218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.811383 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] use_stderr = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.811500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] standard_threads = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.811619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] use_syslog = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.811731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.811842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.811967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.812070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.812187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.812299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.812418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] use_journal = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.812418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] use_json = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.812644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.812757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] use_stderr = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.812868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.812988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] use_syslog = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.813107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.813235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 11:22:36.813359 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.813481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.813594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.813705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.813815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.813932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.814055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.814169 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.814287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.814400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.814400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.814617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.814617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.814804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.814804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.815017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.815119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.815304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.815420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.815539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.815652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.815763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.815880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.816001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.816114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.816231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.816343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.816454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.816571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.816692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.816810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.816929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.817045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.817182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.817307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.817431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.817546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.817659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.817770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.817887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.818005 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.818128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.818247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.818366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.818484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.818597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.818709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.818828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.818948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.819060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.819189 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.819296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.819405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.819516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.819613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.819708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.819801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.819904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.819999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.820101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.820203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.820317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.820430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.820534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.820654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.820768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.820895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.821009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.821121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.821244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.821366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.821480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.821591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.821709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.821830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.822430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.822543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.822655 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.822776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.822897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823759 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.823960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.824058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.824160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.824255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.824349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.824443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.824553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.824666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.824764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.824858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.824959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.825061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.825208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.825312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.833436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.833558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.833641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.833721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.833800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.833872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.833942 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834403 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.834935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835176 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.835983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.836959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.837034 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.837105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.837253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.837334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.837450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.837533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.837644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.838288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.838397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.838477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.838595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.838671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.838742 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.838813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.838892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.838966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.838966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839283 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.839997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840143 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840222 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840300 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.840938 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841085 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841622 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.external = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841833 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] auth.external = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.841980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.password = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842501 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] auth.password = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842579 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] auth.token = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] auth.token = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.842948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843171 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] credential.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.843965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.844935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845519 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845590 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845590 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.845942 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] federation.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.846949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847172 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.847988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848718 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.848941 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.849012 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.849182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.849292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.849435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.849543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.849687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.849767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.849922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.850002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.850157 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.850238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.850391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.850470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.850640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.850732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.850891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.850978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.851154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.851235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.851394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.851480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.851644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.851723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.851892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.851969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.852133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.852219 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.852366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.852445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.852599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.852676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.852828 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.852913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.853058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.853214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.853342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.853422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.853580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.853670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.853821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.853899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.854050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.854127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.854292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.854379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.854539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.854617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.854773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.854886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.855011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.855141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.855265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.855383 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.855497 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.855610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.855723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.855846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.855961 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.856075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.856189 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.856303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.856417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.856540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.856655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.856770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.856883 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.856996 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.857110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.857253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.857371 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.857487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.857600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.857747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.857895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.858019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.858141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.858256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.858370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.858517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.858634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.858757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.858873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.858987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.859108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.859225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.859357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.859484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.859610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.859724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.859838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.859951 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.860065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.860187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.860304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.860418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.860531 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.860646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.860787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.860904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.861003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.861099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.861670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.861777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.861875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.879404 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.879556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.879649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.879728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.879803 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.879878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.879969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880194 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880358 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.880953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.881973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882046 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882497 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.882944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883586 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] resource.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.883950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.884030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.884106 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.884178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.884249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.884321 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.884896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.884896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] role.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] role.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.885951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] role.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886321 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] role.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886920 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.886992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887065 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.887957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888038 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888038 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.888980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889323 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889502 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.889962 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890413 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.890964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] token.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.891968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892041 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892347 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.892951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893338 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.connection = **** {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.893959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.connection = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894711 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.894980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895061 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895512 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895879 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.895960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896180 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80434) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 11:22:36.896562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 11:22:36.896562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 11:22:36.918747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.918976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.919205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.919351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.919589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.919817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.927799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.928458 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.957103 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.957394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.958423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:36.958774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.103698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.103910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.154134 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.154363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.155298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.155538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.226530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.226743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.226833 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.226962 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.227054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.227181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.227285 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.227398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.227483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.227613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.227698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.227826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.260759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.260933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.261099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.261216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.261284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.261381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.261461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.261565 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.261650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.261812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.261906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.262025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.262110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.262306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.273261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.274006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.286545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.286768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.286988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.287074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.287204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.287308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.287419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.287505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.287632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.287716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.287844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80434) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.287929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.288089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.288309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 11:22:37.304844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.315468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 11:22:37.315468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.315468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 11:22:37.315681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.315681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 11:22:37.315814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.315814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.315814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.316013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.316087 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.316160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 11:22:37.316160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.316160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.316160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.316160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.317984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.318253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.318519 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.318805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.319532 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.319810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.320084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.320367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.327527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.329175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.331061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.331332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.331605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.331879 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.332152 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.332405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.332672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.332755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.332937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.333022 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.333219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.333305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.333487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.333583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.333749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.333832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.334233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.334503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.334778 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.335049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.335317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.335583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.360187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.361757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.362030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.362265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.364050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.364327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.367255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.367528 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.367799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.370258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.370535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.370843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.375705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.381006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.381108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.381108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.381270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.381270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.381270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.381270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.381653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.381810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.382073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.382332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.383183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.383468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.383748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.384029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.384304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.384634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.384905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.385191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.401521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.401806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.405919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.406210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.406354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.410009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.410296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.410829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.414488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.414783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.415478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.419689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.420631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.423322 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.423619 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.423904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.426646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.426732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.426799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.426799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.428004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.428304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.428602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.428887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.429190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.429476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.444242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.444515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.444779 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.445039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.449137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.449440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.449748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.450014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.452959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.454726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.455289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.458245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.458440 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.458845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.459138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.459427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.459696 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.459943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.460221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.464141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.464423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.464706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.466676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.466676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.473384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.473659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.473935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.474209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.474479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.474762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.475031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.478765 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.479043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.479324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.479611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.479888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.480189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.480469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.494856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.498132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.500561 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.501442 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.503023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.503903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.504787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.505058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.505361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.507272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.508881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.510730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.511011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.511299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.512814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.514453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.517738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.518024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.518300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.518568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.518845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.520584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.523834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.524130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.524415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.524696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.528273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.528554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.528823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.534847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.535083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.535364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.536680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80434) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.536957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.537263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.537537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.537806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.538079 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.538352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.538615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.538978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.539255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.539537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.539805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.540067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.540337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80434) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.543317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 11:22:37.543600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.543882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.544160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.544435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.544718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.544996 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.545290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.545669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.545954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.546230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.546547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.546799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.547071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 11:22:37.578476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561c4475b9c0 pid: 80434 (default app) Jul 14 11:22:37.582155 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bf756-1d24-4761-a905-c6a96e266519 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:37.582441 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bf756-1d24-4761-a905-c6a96e266519 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:37.582716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bf756-1d24-4761-a905-c6a96e266519 None None] PATH_INFO: `/v3` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:37.586016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561c4475b9c0 pid: 80433 (default app) Jul 14 11:22:37.586927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Thu Jul 14 11:22:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:22:46.588547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c4f16-d5eb-47f2-9670-eb5ece500e97 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:46.589079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c4f16-d5eb-47f2-9670-eb5ece500e97 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:46.589179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c4f16-d5eb-47f2-9670-eb5ece500e97 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:46.593460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:46.598952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:46.599350 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:46.599511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:46.668054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: INFO dbcounter [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] Registered counter for database keystone Jul 14 11:22:46.688989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG oslo_db.sqlalchemy.engines [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80434) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 11:22:46.695127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writer thread running {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 11:22:46.870196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG passlib.handlers.bcrypt [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80434) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 11:22:46.870545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG passlib.handlers.bcrypt [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80434) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 11:22:46.926258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:46.927868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:47.049560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 11:22:47.291183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c4f16-d5eb-47f2-9670-eb5ece500e97 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:47.361292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: INFO dbcounter [None req-1c5c4f16-d5eb-47f2-9670-eb5ece500e97 None None] Registered counter for database keystone Jul 14 11:22:47.380712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG oslo_db.sqlalchemy.engines [None req-1c5c4f16-d5eb-47f2-9670-eb5ece500e97 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80433) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 11:22:47.387646 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writer thread running {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 11:22:47.509670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c4f16-d5eb-47f2-9670-eb5ece500e97 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:47.513943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adfdb96-5d66-4ae7-9394-c9cc3cd19233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:47.514600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfdb96-5d66-4ae7-9394-c9cc3cd19233 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:47.514885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfdb96-5d66-4ae7-9394-c9cc3cd19233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:47.515163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfdb96-5d66-4ae7-9394-c9cc3cd19233 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:47.523362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adfdb96-5d66-4ae7-9394-c9cc3cd19233 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:47.577526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3adfdb96-5d66-4ae7-9394-c9cc3cd19233 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 11:22:47.577802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adfdb96-5d66-4ae7-9394-c9cc3cd19233 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:47.588557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Thu Jul 14 11:22:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:22:47.594633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:47.609068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770adc9e-8e59-4283-ae77-422f9592c97b None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:47.613633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd630647-5161-4151-a4e7-f7d112551cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:47.616893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd630647-5161-4151-a4e7-f7d112551cae admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:47.616893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd630647-5161-4151-a4e7-f7d112551cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:47.616893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd630647-5161-4151-a4e7-f7d112551cae admin admin] PATH_INFO: `/v3/projects/da372d98ddc34559ac820df9de6d2798` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:47.616893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd630647-5161-4151-a4e7-f7d112551cae admin admin] RBAC: Authorizing `identity:get_project(project_id=da372d98ddc34559ac820df9de6d2798)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:47.671428 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bd630647-5161-4151-a4e7-f7d112551cae admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 11:22:47.671776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd630647-5161-4151-a4e7-f7d112551cae admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:47.676532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Thu Jul 14 11:22:47 2022] GET /identity/v3/projects/da372d98ddc34559ac820df9de6d2798 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:22:49.653175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e16efe-bdc1-429e-bca4-172cb05579be None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:49.653732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e16efe-bdc1-429e-bca4-172cb05579be None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:49.653732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e16efe-bdc1-429e-bca4-172cb05579be None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:49.657849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:22:49.663369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0505a2-a647-4739-b3be-57d951793693 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:49.663795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0505a2-a647-4739-b3be-57d951793693 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:49.663922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0505a2-a647-4739-b3be-57d951793693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:49.682379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-0c0505a2-a647-4739-b3be-57d951793693 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:49.683890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0c0505a2-a647-4739-b3be-57d951793693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:49.691730 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 11:22:49.714159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e16efe-bdc1-429e-bca4-172cb05579be None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:49.726261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e16efe-bdc1-429e-bca4-172cb05579be None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:49.730473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b006a1-3ab6-41eb-a8c2-bb868b2ac3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:49.731105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b006a1-3ab6-41eb-a8c2-bb868b2ac3cc admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:49.731385 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b006a1-3ab6-41eb-a8c2-bb868b2ac3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:49.731659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b006a1-3ab6-41eb-a8c2-bb868b2ac3cc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:49.739266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b006a1-3ab6-41eb-a8c2-bb868b2ac3cc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:49.741044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b006a1-3ab6-41eb-a8c2-bb868b2ac3cc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:49.749828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Thu Jul 14 11:22:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:22:49.755170 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0505a2-a647-4739-b3be-57d951793693 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:49.764846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0505a2-a647-4739-b3be-57d951793693 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:49.769393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e19d7-5cf2-498d-8453-1c38621714a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:49.770249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e19d7-5cf2-498d-8453-1c38621714a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:49.770532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e19d7-5cf2-498d-8453-1c38621714a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:49.770808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e19d7-5cf2-498d-8453-1c38621714a8 admin admin] PATH_INFO: `/v3/users/e0d5b9edc140499bb612222f6572ae20` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:49.772215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e19d7-5cf2-498d-8453-1c38621714a8 admin admin] RBAC: Authorizing `identity:get_user(user_id=e0d5b9edc140499bb612222f6572ae20)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:49.774102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e19d7-5cf2-498d-8453-1c38621714a8 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:49.776276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Thu Jul 14 11:22:49 2022] GET /identity/v3/users/e0d5b9edc140499bb612222f6572ae20 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:22:51.887353 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a969-00dc-4313-abad-e6c34787eb69 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:51.887896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a969-00dc-4313-abad-e6c34787eb69 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:51.887896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a969-00dc-4313-abad-e6c34787eb69 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:51.889269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:51.893828 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747513a-35f1-4c92-ac38-ce564c28a4b0 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:51.894252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747513a-35f1-4c92-ac38-ce564c28a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:51.894349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acca91-f9e7-42c7-b5c2-f5022b189383 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:51.894437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747513a-35f1-4c92-ac38-ce564c28a4b0 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:51.894517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acca91-f9e7-42c7-b5c2-f5022b189383 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:51.894752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acca91-f9e7-42c7-b5c2-f5022b189383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:51.895733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:51.900516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06e880-a909-4543-8a78-ddbb62bc3741 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:51.900876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06e880-a909-4543-8a78-ddbb62bc3741 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:51.901173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06e880-a909-4543-8a78-ddbb62bc3741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:51.923893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-0e06e880-a909-4543-8a78-ddbb62bc3741 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:51.927675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0e06e880-a909-4543-8a78-ddbb62bc3741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:51.936840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG passlib.handlers.bcrypt [None req-e1acca91-f9e7-42c7-b5c2-f5022b189383 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80433) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 11:22:51.937305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG passlib.handlers.bcrypt [None req-e1acca91-f9e7-42c7-b5c2-f5022b189383 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80433) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 11:22:51.983056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e1acca91-f9e7-42c7-b5c2-f5022b189383 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:51.984659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e1acca91-f9e7-42c7-b5c2-f5022b189383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:52.011363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 7/12] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:22:52.034502 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06e880-a909-4543-8a78-ddbb62bc3741 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:52.059539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:52.083887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 6/13] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:22:52.105937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acca91-f9e7-42c7-b5c2-f5022b189383 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:52.145442 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06e880-a909-4543-8a78-ddbb62bc3741 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:52.178475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0375296-f14d-43c6-8688-4ae9f718735c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:52.179258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0375296-f14d-43c6-8688-4ae9f718735c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:52.179537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0375296-f14d-43c6-8688-4ae9f718735c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:52.179809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0375296-f14d-43c6-8688-4ae9f718735c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:52.191311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0375296-f14d-43c6-8688-4ae9f718735c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-a0375296-f14d-43c6-8688-4ae9f718735c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:22:52.202363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:22:52.203508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:22:52.204746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 8/14] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:22:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:22:52.210860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acca91-f9e7-42c7-b5c2-f5022b189383 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:52.211001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0375296-f14d-43c6-8688-4ae9f718735c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:52.214285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:52.248331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514ea6c-12ac-4b43-8173-d7691ee7bc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:52.249220 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514ea6c-12ac-4b43-8173-d7691ee7bc21 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:52.249513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514ea6c-12ac-4b43-8173-d7691ee7bc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:52.249789 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514ea6c-12ac-4b43-8173-d7691ee7bc21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:52.293106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:52.310776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0375296-f14d-43c6-8688-4ae9f718735c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:52.318357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514ea6c-12ac-4b43-8173-d7691ee7bc21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:52.319897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514ea6c-12ac-4b43-8173-d7691ee7bc21 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:52.344738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd378f96-64e4-451b-9dd7-946baaabed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:52.345438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd378f96-64e4-451b-9dd7-946baaabed75 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:52.345736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd378f96-64e4-451b-9dd7-946baaabed75 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:52.346013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd378f96-64e4-451b-9dd7-946baaabed75 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:52.346773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd378f96-64e4-451b-9dd7-946baaabed75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:22:52.347081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd378f96-64e4-451b-9dd7-946baaabed75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:52.348482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd378f96-64e4-451b-9dd7-946baaabed75 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:52.361082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 9/15] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:22:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:22:52.366547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd378f96-64e4-451b-9dd7-946baaabed75 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:52.419355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 7/16] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:22:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 11:22:52.424717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514ea6c-12ac-4b43-8173-d7691ee7bc21 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:52.428071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:52.467457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd378f96-64e4-451b-9dd7-946baaabed75 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:52.502317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8872d31-4801-43fd-8d15-47a51afbe47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:52.503187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872d31-4801-43fd-8d15-47a51afbe47f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:52.503471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872d31-4801-43fd-8d15-47a51afbe47f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:52.503744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872d31-4801-43fd-8d15-47a51afbe47f None admin] PATH_INFO: `/v3/users/e0d5b9edc140499bb612222f6572ae20` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:52.504996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8872d31-4801-43fd-8d15-47a51afbe47f None admin] RBAC: Authorizing `identity:get_user(user_id=e0d5b9edc140499bb612222f6572ae20)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:52.506500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8872d31-4801-43fd-8d15-47a51afbe47f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:52.508198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 10/17] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:22:52 2022] GET /identity/v3/users/e0d5b9edc140499bb612222f6572ae20 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:22:52.514002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8872d31-4801-43fd-8d15-47a51afbe47f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:52.522876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514ea6c-12ac-4b43-8173-d7691ee7bc21 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:52.528356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:52.557051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a6cdb-ffbc-4420-8467-15909d621ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:52.557873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a6cdb-ffbc-4420-8467-15909d621ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:52.558160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a6cdb-ffbc-4420-8467-15909d621ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:52.558438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a6cdb-ffbc-4420-8467-15909d621ef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:52.578849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a6cdb-ffbc-4420-8467-15909d621ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:52.580506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a6cdb-ffbc-4420-8467-15909d621ef7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-bf4a6cdb-ffbc-4420-8467-15909d621ef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:22:52.600939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:22:52.601892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:22:52.603294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:22:52.603294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:22:52.603294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:22:52.603294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:22:52.603294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 8/18] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:22:52.609354 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a6cdb-ffbc-4420-8467-15909d621ef7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:52.613736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8872d31-4801-43fd-8d15-47a51afbe47f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:52.649175 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1bf4f-f22b-4e7c-bbff-5dd7a82da740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:52.652662 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1bf4f-f22b-4e7c-bbff-5dd7a82da740 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:52.652662 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1bf4f-f22b-4e7c-bbff-5dd7a82da740 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:52.653020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1bf4f-f22b-4e7c-bbff-5dd7a82da740 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:52.655419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1bf4f-f22b-4e7c-bbff-5dd7a82da740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:52.657038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1bf4f-f22b-4e7c-bbff-5dd7a82da740 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:52.658872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 11/19] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:22:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:22:52.664313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1bf4f-f22b-4e7c-bbff-5dd7a82da740 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:52.668471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:52.712667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a6cdb-ffbc-4420-8467-15909d621ef7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:52.747549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34bc4b-6bf8-4c62-a356-01c576cad7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:52.748219 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34bc4b-6bf8-4c62-a356-01c576cad7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:52.748366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:52.748503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34bc4b-6bf8-4c62-a356-01c576cad7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:52.748786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34bc4b-6bf8-4c62-a356-01c576cad7a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:52.749562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34bc4b-6bf8-4c62-a356-01c576cad7a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:22:52.749874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34bc4b-6bf8-4c62-a356-01c576cad7a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:52.751298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34bc4b-6bf8-4c62-a356-01c576cad7a4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:52.766801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1bf4f-f22b-4e7c-bbff-5dd7a82da740 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:52.768680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 9/20] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:22:52.807563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee974e-66ab-4153-830b-5d4a2c3abac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:52.808231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee974e-66ab-4153-830b-5d4a2c3abac6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:52.808515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee974e-66ab-4153-830b-5d4a2c3abac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:52.808790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee974e-66ab-4153-830b-5d4a2c3abac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:52.809707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee974e-66ab-4153-830b-5d4a2c3abac6 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.domain.id=default, user.id=e0d5b9edc140499bb612222f6572ae20) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:22:52.810018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee974e-66ab-4153-830b-5d4a2c3abac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:52.811516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee974e-66ab-4153-830b-5d4a2c3abac6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:52.819280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 12/21] 10.222.0.40 () {56 vars in 1322 bytes} [Thu Jul 14 11:22:52 2022] GET /identity/v3/role_assignments?user.id=e0d5b9edc140499bb612222f6572ae20&scope.domain.id=default&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:22:55.185350 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90efc01-baae-486f-ac32-03ae81bc76b6 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:55.185844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90efc01-baae-486f-ac32-03ae81bc76b6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:55.185844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90efc01-baae-486f-ac32-03ae81bc76b6 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:55.187186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 10/22] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:55.190531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a63301-1943-4bd3-bf18-bda37bd2e6e3 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:55.191269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a63301-1943-4bd3-bf18-bda37bd2e6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:55.191763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a63301-1943-4bd3-bf18-bda37bd2e6e3 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:55.193359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 13/23] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:55.194775 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26ed4d-f672-4a51-8ee4-5cdc3d61687d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:55.195278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26ed4d-f672-4a51-8ee4-5cdc3d61687d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:55.195779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26ed4d-f672-4a51-8ee4-5cdc3d61687d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:55.203384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251826dc-a6c4-4364-a25b-7b561613ea83 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:55.203993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251826dc-a6c4-4364-a25b-7b561613ea83 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:55.204727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251826dc-a6c4-4364-a25b-7b561613ea83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:55.233743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4a26ed4d-f672-4a51-8ee4-5cdc3d61687d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:55.234985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-251826dc-a6c4-4364-a25b-7b561613ea83 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:55.236399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-251826dc-a6c4-4364-a25b-7b561613ea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:55.237791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4a26ed4d-f672-4a51-8ee4-5cdc3d61687d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:55.368601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 11/24] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:22:55.374997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050b7e8-0444-4d7d-b5ec-501acf508d9b None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:55.375298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050b7e8-0444-4d7d-b5ec-501acf508d9b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:55.375582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050b7e8-0444-4d7d-b5ec-501acf508d9b None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:55.376923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 12/25] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:22:55.382293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c05cb5-7576-4950-8173-ac83bb55fd2e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:55.382582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c05cb5-7576-4950-8173-ac83bb55fd2e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:55.382862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c05cb5-7576-4950-8173-ac83bb55fd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:55.384218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 14/26] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 14 11:22:55.408069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d9c05cb5-7576-4950-8173-ac83bb55fd2e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:55.409348 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d9c05cb5-7576-4950-8173-ac83bb55fd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:55.410574 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251826dc-a6c4-4364-a25b-7b561613ea83 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:55.462137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:55.479572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 13/27] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 14 11:22:55.480423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c05cb5-7576-4950-8173-ac83bb55fd2e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:55.578018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:55.583890 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c05cb5-7576-4950-8173-ac83bb55fd2e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:55.619640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88494528-57f7-420f-8dc3-913c587bedd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:55.620443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494528-57f7-420f-8dc3-913c587bedd1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:55.620728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494528-57f7-420f-8dc3-913c587bedd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:55.621007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494528-57f7-420f-8dc3-913c587bedd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:55.632724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88494528-57f7-420f-8dc3-913c587bedd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:55.633436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251826dc-a6c4-4364-a25b-7b561613ea83 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:55.635059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88494528-57f7-420f-8dc3-913c587bedd1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-88494528-57f7-420f-8dc3-913c587bedd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:22:55.655620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:22:55.656514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:22:55.657364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:22:55.657364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:22:55.657364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:22:55.657364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:22:55.657364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 14/28] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:22:55.657640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88494528-57f7-420f-8dc3-913c587bedd1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:55.693592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:55.726519 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5ff60-d914-4cd4-8186-b2793450d1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:55.727611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5ff60-d914-4cd4-8186-b2793450d1ec None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:55.728044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5ff60-d914-4cd4-8186-b2793450d1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:55.728480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5ff60-d914-4cd4-8186-b2793450d1ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:55.761780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5ff60-d914-4cd4-8186-b2793450d1ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-59a5ff60-d914-4cd4-8186-b2793450d1ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:22:55.769731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:22:55.770773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:22:55.771713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:22:55.781185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88494528-57f7-420f-8dc3-913c587bedd1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:55.782827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 15/29] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:22:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:22:55.783770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5ff60-d914-4cd4-8186-b2793450d1ec None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:55.816309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9ab67-a356-484b-a854-b11bde621815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:55.819080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9ab67-a356-484b-a854-b11bde621815 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:55.819372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9ab67-a356-484b-a854-b11bde621815 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:55.819651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9ab67-a356-484b-a854-b11bde621815 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:55.820425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9ab67-a356-484b-a854-b11bde621815 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:55.822380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9ab67-a356-484b-a854-b11bde621815 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:55.920552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5ff60-d914-4cd4-8186-b2793450d1ec None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:55.957008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe435a-cd55-4ab5-b3d2-bf0fcc9a11a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:55.957797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe435a-cd55-4ab5-b3d2-bf0fcc9a11a6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:55.958086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe435a-cd55-4ab5-b3d2-bf0fcc9a11a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:55.958362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe435a-cd55-4ab5-b3d2-bf0fcc9a11a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:55.959157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe435a-cd55-4ab5-b3d2-bf0fcc9a11a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:22:55.959469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe435a-cd55-4ab5-b3d2-bf0fcc9a11a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:55.960871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe435a-cd55-4ab5-b3d2-bf0fcc9a11a6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:55.963129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:55.980357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 16/30] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:22:55.981203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe435a-cd55-4ab5-b3d2-bf0fcc9a11a6 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:56.089211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe435a-cd55-4ab5-b3d2-bf0fcc9a11a6 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:56.129206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c426020-5e5c-4ace-970e-acf605e94385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:56.129206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c426020-5e5c-4ace-970e-acf605e94385 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:56.129206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c426020-5e5c-4ace-970e-acf605e94385 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:56.129206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c426020-5e5c-4ace-970e-acf605e94385 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:56.129647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c426020-5e5c-4ace-970e-acf605e94385 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:22:56.129647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c426020-5e5c-4ace-970e-acf605e94385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:56.129647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c426020-5e5c-4ace-970e-acf605e94385 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:56.135563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 17/31] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:22:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:22:56.136511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c426020-5e5c-4ace-970e-acf605e94385 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:56.189614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 15/32] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jul 14 11:22:55 2022] POST /identity/v3/roles => generated 219 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:22:56.191500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9ab67-a356-484b-a854-b11bde621815 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:56.208749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:56.314920 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c426020-5e5c-4ace-970e-acf605e94385 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:56.381559 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:56.384582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6845f-d6dc-4146-89eb-a849b2e1c4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:56.385287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6845f-d6dc-4146-89eb-a849b2e1c4f5 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:56.385576 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6845f-d6dc-4146-89eb-a849b2e1c4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:56.385853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6845f-d6dc-4146-89eb-a849b2e1c4f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:56.386586 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6845f-d6dc-4146-89eb-a849b2e1c4f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:56.388080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6845f-d6dc-4146-89eb-a849b2e1c4f5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:56.396932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9ab67-a356-484b-a854-b11bde621815 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:56.439636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c37628-f9a3-41fc-9400-36546e6817d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:56.440631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c37628-f9a3-41fc-9400-36546e6817d9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:56.440930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c37628-f9a3-41fc-9400-36546e6817d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:56.441235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c37628-f9a3-41fc-9400-36546e6817d9 None admin] PATH_INFO: `/v3/users/e0d5b9edc140499bb612222f6572ae20` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:56.443164 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c37628-f9a3-41fc-9400-36546e6817d9 None admin] RBAC: Authorizing `identity:get_user(user_id=e0d5b9edc140499bb612222f6572ae20)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:56.444785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c37628-f9a3-41fc-9400-36546e6817d9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:56.447368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 16/33] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:22:56 2022] GET /identity/v3/users/e0d5b9edc140499bb612222f6572ae20 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:22:56.448944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b24854-7748-4e79-9520-1838815ea855 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:56.449260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b24854-7748-4e79-9520-1838815ea855 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:56.449544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b24854-7748-4e79-9520-1838815ea855 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:56.450916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 17/34] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:56.452940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b24854-7748-4e79-9520-1838815ea855 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:56.556675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b24854-7748-4e79-9520-1838815ea855 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:56.595235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edbdcb-05dd-47f2-a264-f02e6b6ebf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:56.595961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edbdcb-05dd-47f2-a264-f02e6b6ebf36 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:56.596248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edbdcb-05dd-47f2-a264-f02e6b6ebf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:56.596525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edbdcb-05dd-47f2-a264-f02e6b6ebf36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:56.600233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13edbdcb-05dd-47f2-a264-f02e6b6ebf36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:56.601733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13edbdcb-05dd-47f2-a264-f02e6b6ebf36 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:56.608036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 18/35] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:22:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:22:56.608036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8fc62-574f-4a14-9445-000f3e910b14 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:56.608036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8fc62-574f-4a14-9445-000f3e910b14 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:56.608036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8fc62-574f-4a14-9445-000f3e910b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:56.625125 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c1c8fc62-574f-4a14-9445-000f3e910b14 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:56.626683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c1c8fc62-574f-4a14-9445-000f3e910b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:56.646274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:56.703718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 19/36] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:22:56.705117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37894afc-eddc-4753-858f-8e3a064cf2e6 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:56.705440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37894afc-eddc-4753-858f-8e3a064cf2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:56.705737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37894afc-eddc-4753-858f-8e3a064cf2e6 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:56.711197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 20/37] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:56.711197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37894afc-eddc-4753-858f-8e3a064cf2e6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:56.768544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 18/38] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:22:56 2022] POST /identity/v3/projects => generated 301 bytes in 632 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:22:56.770041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8c20e-8679-4e94-ac7f-2b57214143ea None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:56.770345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8c20e-8679-4e94-ac7f-2b57214143ea None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:56.770613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8c20e-8679-4e94-ac7f-2b57214143ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:56.790284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-bab8c20e-8679-4e94-ac7f-2b57214143ea None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:56.791904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-bab8c20e-8679-4e94-ac7f-2b57214143ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:56.852599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37894afc-eddc-4753-858f-8e3a064cf2e6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:56.877491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 19/39] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:22:56.877491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8c20e-8679-4e94-ac7f-2b57214143ea None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:56.894215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433b123-5c2e-40b0-acdc-3f5bfbe25e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:56.895116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433b123-5c2e-40b0-acdc-3f5bfbe25e15 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:56.895618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433b123-5c2e-40b0-acdc-3f5bfbe25e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:56.895724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433b123-5c2e-40b0-acdc-3f5bfbe25e15 None admin] PATH_INFO: `/v3/domains/default/users/e0d5b9edc140499bb612222f6572ae20/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:56.897891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433b123-5c2e-40b0-acdc-3f5bfbe25e15 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e0d5b9edc140499bb612222f6572ae20, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:56.899419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433b123-5c2e-40b0-acdc-3f5bfbe25e15 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:56.926995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:56.993579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8c20e-8679-4e94-ac7f-2b57214143ea None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:57.030218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77682cde-b48e-4dee-9421-97dd8d32f752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:57.030756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77682cde-b48e-4dee-9421-97dd8d32f752 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:57.031044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77682cde-b48e-4dee-9421-97dd8d32f752 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:57.031325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77682cde-b48e-4dee-9421-97dd8d32f752 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:57.032089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77682cde-b48e-4dee-9421-97dd8d32f752 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:57.033656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77682cde-b48e-4dee-9421-97dd8d32f752 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:57.043810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 21/40] 10.222.0.40 () {58 vars in 1361 bytes} [Thu Jul 14 11:22:56 2022] PUT /identity/v3/domains/default/users/e0d5b9edc140499bb612222f6572ae20/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:22:57.045298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f698d-8177-4afe-9a13-e306bffe1fdd None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:57.045780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f698d-8177-4afe-9a13-e306bffe1fdd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:57.046073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f698d-8177-4afe-9a13-e306bffe1fdd None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:57.047575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 22/41] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:57.048204 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f698d-8177-4afe-9a13-e306bffe1fdd None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:57.067496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:57.153841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 20/42] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jul 14 11:22:56 2022] POST /identity/v3/roles => generated 225 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:22:57.155186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8f51-528c-4113-9b4e-6dca48f479b7 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:57.155511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8f51-528c-4113-9b4e-6dca48f479b7 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:57.155988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8f51-528c-4113-9b4e-6dca48f479b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:57.167831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f698d-8177-4afe-9a13-e306bffe1fdd None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:57.188873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-fa6c8f51-528c-4113-9b4e-6dca48f479b7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:57.190389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-fa6c8f51-528c-4113-9b4e-6dca48f479b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:57.214318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcb133-f244-469f-8997-9c3369de3894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:57.215201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcb133-f244-469f-8997-9c3369de3894 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:57.215492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcb133-f244-469f-8997-9c3369de3894 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:57.215774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcb133-f244-469f-8997-9c3369de3894 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:57.217147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcb133-f244-469f-8997-9c3369de3894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:57.218685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcb133-f244-469f-8997-9c3369de3894 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:57.220844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 23/43] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:22:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:22:57.227612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcb133-f244-469f-8997-9c3369de3894 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:57.277850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 21/44] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:22:57.301623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8f51-528c-4113-9b4e-6dca48f479b7 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:57.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:57.375738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:57.378777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcb133-f244-469f-8997-9c3369de3894 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:57.423922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9002d4e-9f36-40f4-9d13-bd2ee1cf6e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:57.424990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9002d4e-9f36-40f4-9d13-bd2ee1cf6e92 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:57.425530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9002d4e-9f36-40f4-9d13-bd2ee1cf6e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:57.425986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9002d4e-9f36-40f4-9d13-bd2ee1cf6e92 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:57.426895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9002d4e-9f36-40f4-9d13-bd2ee1cf6e92 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:57.428598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9002d4e-9f36-40f4-9d13-bd2ee1cf6e92 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:57.436851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8f51-528c-4113-9b4e-6dca48f479b7 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:57.471994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff84be-af28-4a0b-8b3d-93d5a04f20fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:57.472667 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff84be-af28-4a0b-8b3d-93d5a04f20fc None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:57.472952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff84be-af28-4a0b-8b3d-93d5a04f20fc None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:57.473255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff84be-af28-4a0b-8b3d-93d5a04f20fc None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:57.474019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff84be-af28-4a0b-8b3d-93d5a04f20fc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:57.475410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff84be-af28-4a0b-8b3d-93d5a04f20fc None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:57.582864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 24/45] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:22:57 2022] POST /identity/v3/projects => generated 312 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:22:57.628010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 22/46] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jul 14 11:22:57 2022] POST /identity/v3/roles => generated 223 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:22:59.171113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488f13-78f6-4c72-b8c9-f5c1bb29121f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:59.171629 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488f13-78f6-4c72-b8c9-f5c1bb29121f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:59.171629 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488f13-78f6-4c72-b8c9-f5c1bb29121f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:59.172972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 25/47] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:59.178178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c1dd2-0803-408d-be3d-b696679fc4a6 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:59.178579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c1dd2-0803-408d-be3d-b696679fc4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:59.178876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c1dd2-0803-408d-be3d-b696679fc4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:59.197379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-136c1dd2-0803-408d-be3d-b696679fc4a6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:59.198576 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-136c1dd2-0803-408d-be3d-b696679fc4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:59.265608 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 23/48] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:22:59.288510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93488f13-78f6-4c72-b8c9-f5c1bb29121f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:59.391193 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93488f13-78f6-4c72-b8c9-f5c1bb29121f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:59.427283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61174e1e-95cf-4b10-b31c-943a2fb1b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:59.427991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61174e1e-95cf-4b10-b31c-943a2fb1b42a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:59.428276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61174e1e-95cf-4b10-b31c-943a2fb1b42a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:59.428556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61174e1e-95cf-4b10-b31c-943a2fb1b42a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:59.431219 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:59.441348 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61174e1e-95cf-4b10-b31c-943a2fb1b42a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-61174e1e-95cf-4b10-b31c-943a2fb1b42a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:22:59.450266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:22:59.451281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:22:59.452215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 26/49] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:22:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:22:59.459226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c1dd2-0803-408d-be3d-b696679fc4a6 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:59.478363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:59.559688 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c1dd2-0803-408d-be3d-b696679fc4a6 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:59.594356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0d5f8-d436-4913-87a9-49b535824179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:59.595035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0d5f8-d436-4913-87a9-49b535824179 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:59.595322 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0d5f8-d436-4913-87a9-49b535824179 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:59.595605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0d5f8-d436-4913-87a9-49b535824179 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:59.596368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0d5f8-d436-4913-87a9-49b535824179 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:22:59.596679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0d5f8-d436-4913-87a9-49b535824179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:59.598106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0d5f8-d436-4913-87a9-49b535824179 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:59.606948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 24/50] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:22:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:22:59.612412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61174e1e-95cf-4b10-b31c-943a2fb1b42a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:59.629348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9c2e9-1e59-431d-b6fc-ff752bf56cc4 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:59.629642 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9c2e9-1e59-431d-b6fc-ff752bf56cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:59.629928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9c2e9-1e59-431d-b6fc-ff752bf56cc4 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:59.631358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 25/51] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:22:59.636227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ff94-0337-4673-80df-1f92d4607dde None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:59.636523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ff94-0337-4673-80df-1f92d4607dde None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:59.636805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ff94-0337-4673-80df-1f92d4607dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:59.652979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c646ff94-0337-4673-80df-1f92d4607dde None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:59.654207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c646ff94-0337-4673-80df-1f92d4607dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:59.725041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 26/52] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:22:59.748898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ff94-0337-4673-80df-1f92d4607dde None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:59.761483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:59.818679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61174e1e-95cf-4b10-b31c-943a2fb1b42a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:59.855338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ee1bd-6c6e-4e8a-8276-9e4ceb1569cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:59.855584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ff94-0337-4673-80df-1f92d4607dde None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:59.856213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ee1bd-6c6e-4e8a-8276-9e4ceb1569cf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:59.856491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ee1bd-6c6e-4e8a-8276-9e4ceb1569cf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:59.856773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ee1bd-6c6e-4e8a-8276-9e4ceb1569cf None admin] PATH_INFO: `/v3/users/e0d5b9edc140499bb612222f6572ae20` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:59.858209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ee1bd-6c6e-4e8a-8276-9e4ceb1569cf None admin] RBAC: Authorizing `identity:get_user(user_id=e0d5b9edc140499bb612222f6572ae20)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:59.859620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ee1bd-6c6e-4e8a-8276-9e4ceb1569cf None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:59.861395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 27/53] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:22:59 2022] GET /identity/v3/users/e0d5b9edc140499bb612222f6572ae20 => generated 247 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:22:59.867012 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ee1bd-6c6e-4e8a-8276-9e4ceb1569cf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:59.892458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dcf79-d020-4b74-8622-848a38db1e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:59.893284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dcf79-d020-4b74-8622-848a38db1e89 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:59.893573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dcf79-d020-4b74-8622-848a38db1e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:59.893855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dcf79-d020-4b74-8622-848a38db1e89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:59.895136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dcf79-d020-4b74-8622-848a38db1e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:59.896616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dcf79-d020-4b74-8622-848a38db1e89 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:59.898382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 27/54] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:22:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:22:59.904377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dcf79-d020-4b74-8622-848a38db1e89 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:59.932808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:59.971062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ee1bd-6c6e-4e8a-8276-9e4ceb1569cf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:59.985533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:00.006429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d7ae5-2f06-4279-a838-f51cd2e7ca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:00.007218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7ae5-2f06-4279-a838-f51cd2e7ca7a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:00.007500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7ae5-2f06-4279-a838-f51cd2e7ca7a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:00.007595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dcf79-d020-4b74-8622-848a38db1e89 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:00.007770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7ae5-2f06-4279-a838-f51cd2e7ca7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:00.008970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d7ae5-2f06-4279-a838-f51cd2e7ca7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:00.010449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d7ae5-2f06-4279-a838-f51cd2e7ca7a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:00.012051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 28/55] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:22:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:00.018042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d7ae5-2f06-4279-a838-f51cd2e7ca7a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:00.043552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b9bf9-26df-48bd-874c-d9ca1e5e275b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:00.044218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b9bf9-26df-48bd-874c-d9ca1e5e275b None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:00.044505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b9bf9-26df-48bd-874c-d9ca1e5e275b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:00.044783 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b9bf9-26df-48bd-874c-d9ca1e5e275b None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:00.045543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b9bf9-26df-48bd-874c-d9ca1e5e275b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:00.047078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b9bf9-26df-48bd-874c-d9ca1e5e275b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:00.116122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d7ae5-2f06-4279-a838-f51cd2e7ca7a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:00.143591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:00.149725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5f197-3d9a-412a-83cb-40ae23e89532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:00.150430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f197-3d9a-412a-83cb-40ae23e89532 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:00.150695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f197-3d9a-412a-83cb-40ae23e89532 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:00.150966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f197-3d9a-412a-83cb-40ae23e89532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:00.151834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5f197-3d9a-412a-83cb-40ae23e89532 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.domain.id=default, user.id=e0d5b9edc140499bb612222f6572ae20) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:00.156030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5f197-3d9a-412a-83cb-40ae23e89532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:00.156030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5f197-3d9a-412a-83cb-40ae23e89532 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:00.161513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 29/56] 10.222.0.40 () {56 vars in 1322 bytes} [Thu Jul 14 11:23:00 2022] GET /identity/v3/role_assignments?user.id=e0d5b9edc140499bb612222f6572ae20&scope.domain.id=default&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:00.183544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 28/57] 10.222.0.40 () {60 vars in 1140 bytes} [Thu Jul 14 11:22:59 2022] POST /identity/v3/projects => generated 298 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:23:02.270344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3e522-fb5b-45b9-9f03-c34b06d2ddff None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:02.270840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3e522-fb5b-45b9-9f03-c34b06d2ddff None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:02.270924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3e522-fb5b-45b9-9f03-c34b06d2ddff None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:02.272275 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 30/58] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:02.277425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab8f04-1819-47e8-bf30-ef3ddca19326 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:02.277840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab8f04-1819-47e8-bf30-ef3ddca19326 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:02.277991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab8f04-1819-47e8-bf30-ef3ddca19326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:02.294161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-afab8f04-1819-47e8-bf30-ef3ddca19326 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:02.295350 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-afab8f04-1819-47e8-bf30-ef3ddca19326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:02.359337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 29/59] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:02.383700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3e522-fb5b-45b9-9f03-c34b06d2ddff None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:02.483200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3e522-fb5b-45b9-9f03-c34b06d2ddff None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:02.516378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370f2a2-c52c-4dc1-af9c-0b8be073c85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:02.517203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370f2a2-c52c-4dc1-af9c-0b8be073c85c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:02.517467 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370f2a2-c52c-4dc1-af9c-0b8be073c85c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:02.517741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370f2a2-c52c-4dc1-af9c-0b8be073c85c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:02.518938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370f2a2-c52c-4dc1-af9c-0b8be073c85c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:02.520454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370f2a2-c52c-4dc1-af9c-0b8be073c85c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:02.522035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 31/60] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:02.527769 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab8f04-1819-47e8-bf30-ef3ddca19326 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:02.585062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:02.624893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab8f04-1819-47e8-bf30-ef3ddca19326 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:02.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f67ac-980c-42ce-b230-98f33d54701f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:02.657966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f67ac-980c-42ce-b230-98f33d54701f None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:02.658256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f67ac-980c-42ce-b230-98f33d54701f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:02.658538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f67ac-980c-42ce-b230-98f33d54701f None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:02.659272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f67ac-980c-42ce-b230-98f33d54701f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:02.660745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f67ac-980c-42ce-b230-98f33d54701f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:02.767839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 30/61] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:02 2022] POST /identity/v3/users => generated 275 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:05.825628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bea1f-13ee-4451-979b-95556b92c147 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:05.826645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bea1f-13ee-4451-979b-95556b92c147 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:05.827127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bea1f-13ee-4451-979b-95556b92c147 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:05.828993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 32/62] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:05.834353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcfa3d-fd23-4123-9786-9b1a7aef67b8 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:05.835210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcfa3d-fd23-4123-9786-9b1a7aef67b8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:05.835709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcfa3d-fd23-4123-9786-9b1a7aef67b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:05.858180 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-90fcfa3d-fd23-4123-9786-9b1a7aef67b8 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:05.859910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-90fcfa3d-fd23-4123-9786-9b1a7aef67b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:05.955867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 31/63] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:05.994929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bea1f-13ee-4451-979b-95556b92c147 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:06.077314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:06.137867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bea1f-13ee-4451-979b-95556b92c147 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:06.182094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab97aa2-89a6-4cc5-af2f-d9371714eb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:06.182899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab97aa2-89a6-4cc5-af2f-d9371714eb70 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.183184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab97aa2-89a6-4cc5-af2f-d9371714eb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.183460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab97aa2-89a6-4cc5-af2f-d9371714eb70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.205192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bc307-64e3-47f2-abea-c6c17b0a9c8f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.205500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bc307-64e3-47f2-abea-c6c17b0a9c8f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.205756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bc307-64e3-47f2-abea-c6c17b0a9c8f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.207170 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 32/64] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:23:06.210008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab97aa2-89a6-4cc5-af2f-d9371714eb70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-1ab97aa2-89a6-4cc5-af2f-d9371714eb70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:06.218351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:06.219201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:06.219952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 33/65] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:23:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:06.223337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33149af5-30e9-4a11-89bf-3c80044116e6 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.223634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33149af5-30e9-4a11-89bf-3c80044116e6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.223920 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33149af5-30e9-4a11-89bf-3c80044116e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.230790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab97aa2-89a6-4cc5-af2f-d9371714eb70 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:06.241727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-33149af5-30e9-4a11-89bf-3c80044116e6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:06.243287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-33149af5-30e9-4a11-89bf-3c80044116e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:06.295220 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:06.325347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 33/66] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:06.334087 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893ea94-4c3f-4f16-9649-60b4c93bb22a None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.334755 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893ea94-4c3f-4f16-9649-60b4c93bb22a None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.335599 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893ea94-4c3f-4f16-9649-60b4c93bb22a None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.343001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 34/67] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:06 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:06.343001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463030b6-5ca8-49ca-8f8f-97e3885ad133 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.343001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463030b6-5ca8-49ca-8f8f-97e3885ad133 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.343273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463030b6-5ca8-49ca-8f8f-97e3885ad133 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.344388 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 35/68] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:06.346287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572fee5-2767-49fa-a9e5-eb6ba3c2fd30 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.349449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572fee5-2767-49fa-a9e5-eb6ba3c2fd30 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.349449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572fee5-2767-49fa-a9e5-eb6ba3c2fd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.355298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab97aa2-89a6-4cc5-af2f-d9371714eb70 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:06.364424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-f572fee5-2767-49fa-a9e5-eb6ba3c2fd30 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:06.365717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f572fee5-2767-49fa-a9e5-eb6ba3c2fd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:06.379523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:06.391326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41200c7d-bf64-4085-a2ad-147e80ee6cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:06.391998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41200c7d-bf64-4085-a2ad-147e80ee6cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.392278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41200c7d-bf64-4085-a2ad-147e80ee6cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.392630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41200c7d-bf64-4085-a2ad-147e80ee6cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.393334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41200c7d-bf64-4085-a2ad-147e80ee6cc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:06.393643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41200c7d-bf64-4085-a2ad-147e80ee6cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:06.395043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41200c7d-bf64-4085-a2ad-147e80ee6cc8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:06.407948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 34/69] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:06.408992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41200c7d-bf64-4085-a2ad-147e80ee6cc8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:06.441939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 36/70] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:06.443443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb380f0-2694-40d7-bb0b-20f0653dedc7 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.443860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb380f0-2694-40d7-bb0b-20f0653dedc7 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.444276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb380f0-2694-40d7-bb0b-20f0653dedc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.461988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-dbb380f0-2694-40d7-bb0b-20f0653dedc7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:06.463424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-dbb380f0-2694-40d7-bb0b-20f0653dedc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:06.531592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41200c7d-bf64-4085-a2ad-147e80ee6cc8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:06.542885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 37/71] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:06.543587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb380f0-2694-40d7-bb0b-20f0653dedc7 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:06.579192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f14e0-f77b-4a2f-b3ae-58fd6cae3610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:06.580424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f14e0-f77b-4a2f-b3ae-58fd6cae3610 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.581187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f14e0-f77b-4a2f-b3ae-58fd6cae3610 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.581876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f14e0-f77b-4a2f-b3ae-58fd6cae3610 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.592135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:06.593951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f14e0-f77b-4a2f-b3ae-58fd6cae3610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-7f8f14e0-f77b-4a2f-b3ae-58fd6cae3610 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:06.601805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:06.607562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 35/72] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:06.608420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f14e0-f77b-4a2f-b3ae-58fd6cae3610 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:06.659365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:06.662667 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb380f0-2694-40d7-bb0b-20f0653dedc7 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:06.702569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c83ec-131a-41ac-a51b-1ebb665b83f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:06.703443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c83ec-131a-41ac-a51b-1ebb665b83f7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.703999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c83ec-131a-41ac-a51b-1ebb665b83f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.704273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c83ec-131a-41ac-a51b-1ebb665b83f7 None admin] PATH_INFO: `/v3/users/e0d5b9edc140499bb612222f6572ae20` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.705513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c83ec-131a-41ac-a51b-1ebb665b83f7 None admin] RBAC: Authorizing `identity:get_user(user_id=e0d5b9edc140499bb612222f6572ae20)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:06.706984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c83ec-131a-41ac-a51b-1ebb665b83f7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:06.713063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 38/73] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:06 2022] GET /identity/v3/users/e0d5b9edc140499bb612222f6572ae20 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:06.713063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c83ec-131a-41ac-a51b-1ebb665b83f7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:06.717380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f14e0-f77b-4a2f-b3ae-58fd6cae3610 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:06.750378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016773d-0530-4811-9dc4-f2016731f7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:06.751170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016773d-0530-4811-9dc4-f2016731f7fb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.751453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016773d-0530-4811-9dc4-f2016731f7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.751730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016773d-0530-4811-9dc4-f2016731f7fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.762898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016773d-0530-4811-9dc4-f2016731f7fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-2016773d-0530-4811-9dc4-f2016731f7fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:06.770588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:06.771464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:06.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 36/74] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:06.772856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016773d-0530-4811-9dc4-f2016731f7fb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:06.819294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c83ec-131a-41ac-a51b-1ebb665b83f7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:06.823396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:06.856986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dede3-2db5-479a-a999-834e9aa1b5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:06.857742 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dede3-2db5-479a-a999-834e9aa1b5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.858024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dede3-2db5-479a-a999-834e9aa1b5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.858300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dede3-2db5-479a-a999-834e9aa1b5c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.871075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dede3-2db5-479a-a999-834e9aa1b5c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:06.877419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016773d-0530-4811-9dc4-f2016731f7fb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-574dede3-2db5-479a-a999-834e9aa1b5c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:06.879104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:06.879990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:06.880762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:06.881409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:06.881476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 39/75] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:06.881539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624af8bd-de5e-4fce-bc2a-4e5172c3e652 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.881821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624af8bd-de5e-4fce-bc2a-4e5172c3e652 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.882451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624af8bd-de5e-4fce-bc2a-4e5172c3e652 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.883500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 40/76] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:06.884090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624af8bd-de5e-4fce-bc2a-4e5172c3e652 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:06.915027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e9885-225f-40de-b47a-38911bded3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:06.915701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e9885-225f-40de-b47a-38911bded3ab None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:06.915992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e9885-225f-40de-b47a-38911bded3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:06.916264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e9885-225f-40de-b47a-38911bded3ab None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:06.917031 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e9885-225f-40de-b47a-38911bded3ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:06.917379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e9885-225f-40de-b47a-38911bded3ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:06.918773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e9885-225f-40de-b47a-38911bded3ab None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:06.927111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 37/77] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:06.927818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e9885-225f-40de-b47a-38911bded3ab None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:06.994299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624af8bd-de5e-4fce-bc2a-4e5172c3e652 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:07.025415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e9885-225f-40de-b47a-38911bded3ab None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:07.026879 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8e46f-9641-4de8-a29c-9b3a40314582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:07.027553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8e46f-9641-4de8-a29c-9b3a40314582 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.027835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8e46f-9641-4de8-a29c-9b3a40314582 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.028113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8e46f-9641-4de8-a29c-9b3a40314582 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.058658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:07.059448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ffdea-5de2-4a9c-ae09-c929c95e2ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:07.060506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ffdea-5de2-4a9c-ae09-c929c95e2ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.060800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ffdea-5de2-4a9c-ae09-c929c95e2ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.061086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ffdea-5de2-4a9c-ae09-c929c95e2ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.061991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ffdea-5de2-4a9c-ae09-c929c95e2ee6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:07.062313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ffdea-5de2-4a9c-ae09-c929c95e2ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:07.063706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ffdea-5de2-4a9c-ae09-c929c95e2ee6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:07.072087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 38/78] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:07.075857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ffdea-5de2-4a9c-ae09-c929c95e2ee6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:07.095298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8e46f-9641-4de8-a29c-9b3a40314582 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:07.097207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8e46f-9641-4de8-a29c-9b3a40314582 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:07.102356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:07.171590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ffdea-5de2-4a9c-ae09-c929c95e2ee6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:07.198303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 41/79] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:07.199950 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65312501-1491-432a-8d77-f125b83f6943 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.200248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65312501-1491-432a-8d77-f125b83f6943 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.200533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65312501-1491-432a-8d77-f125b83f6943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.206508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10442150-37cc-4f3f-be50-fde67f2d0ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:07.211016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10442150-37cc-4f3f-be50-fde67f2d0ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.211016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10442150-37cc-4f3f-be50-fde67f2d0ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.211016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10442150-37cc-4f3f-be50-fde67f2d0ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.211016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10442150-37cc-4f3f-be50-fde67f2d0ff5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:07.211016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10442150-37cc-4f3f-be50-fde67f2d0ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:07.211016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10442150-37cc-4f3f-be50-fde67f2d0ff5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:07.218088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-65312501-1491-432a-8d77-f125b83f6943 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:07.219316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 39/80] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:07.220146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10442150-37cc-4f3f-be50-fde67f2d0ff5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:07.220402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-65312501-1491-432a-8d77-f125b83f6943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:07.290354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:07.299726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 42/81] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:07.311391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65312501-1491-432a-8d77-f125b83f6943 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:07.317151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:07.327046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10442150-37cc-4f3f-be50-fde67f2d0ff5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:07.362967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f303c-109a-42c8-8eb5-7e2f167238e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:07.363839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f303c-109a-42c8-8eb5-7e2f167238e3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.364122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f303c-109a-42c8-8eb5-7e2f167238e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.364400 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f303c-109a-42c8-8eb5-7e2f167238e3 None admin] PATH_INFO: `/v3/users/afd88ca575e04b44b3284ec86097cb2e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.379611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f303c-109a-42c8-8eb5-7e2f167238e3 None admin] RBAC: Authorizing `identity:get_user(user_id=afd88ca575e04b44b3284ec86097cb2e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:07.381496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f303c-109a-42c8-8eb5-7e2f167238e3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:07.383236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 40/82] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:07 2022] GET /identity/v3/users/afd88ca575e04b44b3284ec86097cb2e => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:07.384006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f303c-109a-42c8-8eb5-7e2f167238e3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:07.404034 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65312501-1491-432a-8d77-f125b83f6943 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:07.439725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db1355-2037-43b8-8392-7d5cf7aca3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:07.440514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1355-2037-43b8-8392-7d5cf7aca3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.440797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1355-2037-43b8-8392-7d5cf7aca3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.441072 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1355-2037-43b8-8392-7d5cf7aca3f4 None admin] PATH_INFO: `/v3/users/afd88ca575e04b44b3284ec86097cb2e` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.442286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db1355-2037-43b8-8392-7d5cf7aca3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=afd88ca575e04b44b3284ec86097cb2e)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:07.443749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db1355-2037-43b8-8392-7d5cf7aca3f4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:07.445663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 43/83] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:07 2022] GET /identity/v3/users/afd88ca575e04b44b3284ec86097cb2e => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:07.446227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db1355-2037-43b8-8392-7d5cf7aca3f4 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:07.487670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f303c-109a-42c8-8eb5-7e2f167238e3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:07.506636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:07.520517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fab24-a6a1-4c9d-abfe-8c8493427288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:07.521415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fab24-a6a1-4c9d-abfe-8c8493427288 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.521707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fab24-a6a1-4c9d-abfe-8c8493427288 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.522000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fab24-a6a1-4c9d-abfe-8c8493427288 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.523266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fab24-a6a1-4c9d-abfe-8c8493427288 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:07.525331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fab24-a6a1-4c9d-abfe-8c8493427288 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:07.527050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 41/84] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:07 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:07.527915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fab24-a6a1-4c9d-abfe-8c8493427288 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:07.546239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:07.546853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db1355-2037-43b8-8392-7d5cf7aca3f4 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:07.581329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b35151-da33-4f5f-b472-db3c538ee3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:07.582080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b35151-da33-4f5f-b472-db3c538ee3fd None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.582364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b35151-da33-4f5f-b472-db3c538ee3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.582639 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b35151-da33-4f5f-b472-db3c538ee3fd None admin] PATH_INFO: `/v3/users/afd88ca575e04b44b3284ec86097cb2e` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.583873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b35151-da33-4f5f-b472-db3c538ee3fd None admin] RBAC: Authorizing `identity:get_user(user_id=afd88ca575e04b44b3284ec86097cb2e)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:07.585488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b35151-da33-4f5f-b472-db3c538ee3fd None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:07.587694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 44/85] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:07 2022] GET /identity/v3/users/afd88ca575e04b44b3284ec86097cb2e => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:07.588473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b35151-da33-4f5f-b472-db3c538ee3fd None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:07.634832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fab24-a6a1-4c9d-abfe-8c8493427288 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:07.668180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edd30b-84ba-4ebe-a660-6d2087895d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:07.668969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edd30b-84ba-4ebe-a660-6d2087895d38 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.669307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edd30b-84ba-4ebe-a660-6d2087895d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.669585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edd30b-84ba-4ebe-a660-6d2087895d38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.670760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edd30b-84ba-4ebe-a660-6d2087895d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:07.672210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edd30b-84ba-4ebe-a660-6d2087895d38 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:07.673858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 42/86] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:07.678560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edd30b-84ba-4ebe-a660-6d2087895d38 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:07.690579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b35151-da33-4f5f-b472-db3c538ee3fd None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:07.719250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:07.725312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604d90e-5af6-40d8-be2c-c2ab11974cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:07.725979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604d90e-5af6-40d8-be2c-c2ab11974cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.741559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604d90e-5af6-40d8-be2c-c2ab11974cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.741559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604d90e-5af6-40d8-be2c-c2ab11974cc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.759032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:07.774219 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edd30b-84ba-4ebe-a660-6d2087895d38 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:07.793112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604d90e-5af6-40d8-be2c-c2ab11974cc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:07.794634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604d90e-5af6-40d8-be2c-c2ab11974cc5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:07.807048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4819d1b-e92c-46d5-8530-a7c7ed37c102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:07.807698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4819d1b-e92c-46d5-8530-a7c7ed37c102 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:07.807980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4819d1b-e92c-46d5-8530-a7c7ed37c102 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:07.808256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4819d1b-e92c-46d5-8530-a7c7ed37c102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:07.877284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4819d1b-e92c-46d5-8530-a7c7ed37c102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:07.878771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4819d1b-e92c-46d5-8530-a7c7ed37c102 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:07.894935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 45/87] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:07.895668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604d90e-5af6-40d8-be2c-c2ab11974cc5 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:07.928527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:07.966130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:07.982841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 43/88] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:07.983684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4819d1b-e92c-46d5-8530-a7c7ed37c102 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:07.998762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604d90e-5af6-40d8-be2c-c2ab11974cc5 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:08.032363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8213f-806a-4bdc-a6ad-c40417352773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:08.033020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8213f-806a-4bdc-a6ad-c40417352773 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:08.033342 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8213f-806a-4bdc-a6ad-c40417352773 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:08.033629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8213f-806a-4bdc-a6ad-c40417352773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:08.034836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8213f-806a-4bdc-a6ad-c40417352773 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=d76703b19c6c4521a9480e46863c1979, user.id=e0d5b9edc140499bb612222f6572ae20) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:08.034836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8213f-806a-4bdc-a6ad-c40417352773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:08.036244 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8213f-806a-4bdc-a6ad-c40417352773 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:08.042233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 46/89] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:07 2022] GET /identity/v3/role_assignments?user.id=e0d5b9edc140499bb612222f6572ae20&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:08.043004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8213f-806a-4bdc-a6ad-c40417352773 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:08.100101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4819d1b-e92c-46d5-8530-a7c7ed37c102 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:08.140134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a7d26-5cdd-4042-8dad-b4718cf3e70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:08.140806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a7d26-5cdd-4042-8dad-b4718cf3e70c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:08.141096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a7d26-5cdd-4042-8dad-b4718cf3e70c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:08.141401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a7d26-5cdd-4042-8dad-b4718cf3e70c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:08.144977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:08.153350 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8213f-806a-4bdc-a6ad-c40417352773 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:08.192594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454a3a7-b6da-414e-be61-447ef506e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:08.193622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454a3a7-b6da-414e-be61-447ef506e4bb None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:08.193919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454a3a7-b6da-414e-be61-447ef506e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:08.194195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454a3a7-b6da-414e-be61-447ef506e4bb None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:08.195054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454a3a7-b6da-414e-be61-447ef506e4bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:08.196449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454a3a7-b6da-414e-be61-447ef506e4bb None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:08.206380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:08.229084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a7d26-5cdd-4042-8dad-b4718cf3e70c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:08.229084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a7d26-5cdd-4042-8dad-b4718cf3e70c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:08.320028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 44/90] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:08.320028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a7d26-5cdd-4042-8dad-b4718cf3e70c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:08.341494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 47/91] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:08 2022] POST /identity/v3/users => generated 289 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:08.342660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454a3a7-b6da-414e-be61-447ef506e4bb None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:08.442919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:08.466721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a7d26-5cdd-4042-8dad-b4718cf3e70c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:08.480508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454a3a7-b6da-414e-be61-447ef506e4bb None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:08.508985 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200475f-9c6d-45aa-b5a0-0efb26b0ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:08.509955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200475f-9c6d-45aa-b5a0-0efb26b0ad22 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:08.510239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200475f-9c6d-45aa-b5a0-0efb26b0ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:08.510510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200475f-9c6d-45aa-b5a0-0efb26b0ad22 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:08.511888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200475f-9c6d-45aa-b5a0-0efb26b0ad22 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:08.513179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200475f-9c6d-45aa-b5a0-0efb26b0ad22 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:08.515058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 45/92] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:08 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:08.515743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200475f-9c6d-45aa-b5a0-0efb26b0ad22 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:08.529671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81ca4b-404c-4e41-952c-85c2e8a16d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:08.530982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ca4b-404c-4e41-952c-85c2e8a16d49 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:08.531471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ca4b-404c-4e41-952c-85c2e8a16d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:08.531978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ca4b-404c-4e41-952c-85c2e8a16d49 None admin] PATH_INFO: `/v3/projects/1198d4d7066b4f8c81523f63f0843bd3` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:08.533423 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81ca4b-404c-4e41-952c-85c2e8a16d49 None admin] RBAC: Authorizing `identity:get_project(project_id=1198d4d7066b4f8c81523f63f0843bd3)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:08.535349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81ca4b-404c-4e41-952c-85c2e8a16d49 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:08.537332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 48/93] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:08 2022] GET /identity/v3/projects/1198d4d7066b4f8c81523f63f0843bd3 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:08.538403 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81ca4b-404c-4e41-952c-85c2e8a16d49 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:08.565071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:08.646942 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200475f-9c6d-45aa-b5a0-0efb26b0ad22 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:08.686297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81ca4b-404c-4e41-952c-85c2e8a16d49 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:08.694944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60209881-b7ba-455f-9e0b-ac1aca348e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:08.696439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60209881-b7ba-455f-9e0b-ac1aca348e30 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:08.696728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60209881-b7ba-455f-9e0b-ac1aca348e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:08.697032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60209881-b7ba-455f-9e0b-ac1aca348e30 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:08.699793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60209881-b7ba-455f-9e0b-ac1aca348e30 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:08.701752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60209881-b7ba-455f-9e0b-ac1aca348e30 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:08.704656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 46/94] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:08 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:08.705560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60209881-b7ba-455f-9e0b-ac1aca348e30 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:08.738053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:08.739130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb22e66-b086-44d8-8b83-c17ec29d76fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:08.739986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb22e66-b086-44d8-8b83-c17ec29d76fb None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:08.740277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb22e66-b086-44d8-8b83-c17ec29d76fb None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:08.740617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb22e66-b086-44d8-8b83-c17ec29d76fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:08.741549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb22e66-b086-44d8-8b83-c17ec29d76fb None admin] RBAC: Adding query filter params (role.id=df5932e731294f7498a6983a730628bd, scope.project.id=d76703b19c6c4521a9480e46863c1979, user.id=afd88ca575e04b44b3284ec86097cb2e) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:08.741872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb22e66-b086-44d8-8b83-c17ec29d76fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:08.743982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb22e66-b086-44d8-8b83-c17ec29d76fb None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:08.750647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 49/95] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:08 2022] GET /identity/v3/role_assignments?user.id=afd88ca575e04b44b3284ec86097cb2e&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=df5932e731294f7498a6983a730628bd => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:08.751653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb22e66-b086-44d8-8b83-c17ec29d76fb None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:08.844797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:08.860091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60209881-b7ba-455f-9e0b-ac1aca348e30 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:08.868764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb22e66-b086-44d8-8b83-c17ec29d76fb None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:08.908772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd624b-cb8b-4c84-829e-7e15ccadb1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:08.910133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd624b-cb8b-4c84-829e-7e15ccadb1da None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:08.910495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd624b-cb8b-4c84-829e-7e15ccadb1da None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:08.910847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd624b-cb8b-4c84-829e-7e15ccadb1da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:08.911785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd624b-cb8b-4c84-829e-7e15ccadb1da None admin] RBAC: Adding query filter params (role.id=41fd0da9b16d41c08992073473d87488, scope.project.id=d76703b19c6c4521a9480e46863c1979, user.id=afd88ca575e04b44b3284ec86097cb2e) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:08.912177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd624b-cb8b-4c84-829e-7e15ccadb1da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:08.913770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd624b-cb8b-4c84-829e-7e15ccadb1da None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:08.920932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 50/96] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:08 2022] GET /identity/v3/role_assignments?user.id=afd88ca575e04b44b3284ec86097cb2e&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=41fd0da9b16d41c08992073473d87488 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:08.948213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342bdf0-620a-446b-b1d9-034ea10b42d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:08.949644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342bdf0-620a-446b-b1d9-034ea10b42d3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:08.950105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342bdf0-620a-446b-b1d9-034ea10b42d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:08.950531 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342bdf0-620a-446b-b1d9-034ea10b42d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:08.951566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342bdf0-620a-446b-b1d9-034ea10b42d3 None admin] RBAC: Adding query filter params (role.id=df5932e731294f7498a6983a730628bd, scope.project.id=1198d4d7066b4f8c81523f63f0843bd3, user.id=afd88ca575e04b44b3284ec86097cb2e) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:08.952044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342bdf0-620a-446b-b1d9-034ea10b42d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:08.954060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342bdf0-620a-446b-b1d9-034ea10b42d3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:08.962458 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 47/97] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:08 2022] GET /identity/v3/role_assignments?user.id=afd88ca575e04b44b3284ec86097cb2e&scope.project.id=1198d4d7066b4f8c81523f63f0843bd3&role.id=df5932e731294f7498a6983a730628bd => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:10.356041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f716e-60d0-427b-abbe-d08f9cf678ca None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:10.356559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f716e-60d0-427b-abbe-d08f9cf678ca None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:10.356633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f716e-60d0-427b-abbe-d08f9cf678ca None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:10.358090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 51/98] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:10.363720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39521643-a0c2-418f-b886-bdd5a66f32b3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:10.364150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39521643-a0c2-418f-b886-bdd5a66f32b3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:10.364372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39521643-a0c2-418f-b886-bdd5a66f32b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:10.385273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-39521643-a0c2-418f-b886-bdd5a66f32b3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:10.386481 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-39521643-a0c2-418f-b886-bdd5a66f32b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:10.475131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 48/99] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:10.509349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f716e-60d0-427b-abbe-d08f9cf678ca None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:10.625029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f716e-60d0-427b-abbe-d08f9cf678ca None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:10.662052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668fb27-6644-4b0e-a34a-155f81439a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:10.662870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668fb27-6644-4b0e-a34a-155f81439a03 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:10.663155 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668fb27-6644-4b0e-a34a-155f81439a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:10.663434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668fb27-6644-4b0e-a34a-155f81439a03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:10.675333 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668fb27-6644-4b0e-a34a-155f81439a03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-7668fb27-6644-4b0e-a34a-155f81439a03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:10.683683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:10.684614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:10.685426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 52/100] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:23:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:10.690944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39521643-a0c2-418f-b886-bdd5a66f32b3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:10.710260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:10.820028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39521643-a0c2-418f-b886-bdd5a66f32b3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:10.855544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c9c1a-d4ad-48c4-9de3-0b8096e4ab39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:10.856229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c9c1a-d4ad-48c4-9de3-0b8096e4ab39 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:10.856504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c9c1a-d4ad-48c4-9de3-0b8096e4ab39 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:10.856787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c9c1a-d4ad-48c4-9de3-0b8096e4ab39 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:10.857580 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c9c1a-d4ad-48c4-9de3-0b8096e4ab39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:10.857945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c9c1a-d4ad-48c4-9de3-0b8096e4ab39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:10.859338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c9c1a-d4ad-48c4-9de3-0b8096e4ab39 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:10.874568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 49/101] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:10.880360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668fb27-6644-4b0e-a34a-155f81439a03 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:10.921554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:11.033038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668fb27-6644-4b0e-a34a-155f81439a03 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:11.072186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c9738-3ba6-4371-90a6-d2c787096a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:11.073457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c9738-3ba6-4371-90a6-d2c787096a29 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:11.073991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c9738-3ba6-4371-90a6-d2c787096a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:11.074463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c9738-3ba6-4371-90a6-d2c787096a29 None admin] PATH_INFO: `/v3/users/e0d5b9edc140499bb612222f6572ae20` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:11.075915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c9738-3ba6-4371-90a6-d2c787096a29 None admin] RBAC: Authorizing `identity:get_user(user_id=e0d5b9edc140499bb612222f6572ae20)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:11.077676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c9738-3ba6-4371-90a6-d2c787096a29 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:11.079797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 53/102] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:10 2022] GET /identity/v3/users/e0d5b9edc140499bb612222f6572ae20 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:11.085725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c9c1a-d4ad-48c4-9de3-0b8096e4ab39 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:11.185550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:11.217419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c9c1a-d4ad-48c4-9de3-0b8096e4ab39 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:11.259969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467de574-cdf6-4332-af8b-d3e741c6e697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:11.260846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467de574-cdf6-4332-af8b-d3e741c6e697 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:11.261134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467de574-cdf6-4332-af8b-d3e741c6e697 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:11.261434 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467de574-cdf6-4332-af8b-d3e741c6e697 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:11.262699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467de574-cdf6-4332-af8b-d3e741c6e697 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:11.264152 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467de574-cdf6-4332-af8b-d3e741c6e697 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:11.266200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 50/103] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:11 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:11.271965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c9738-3ba6-4371-90a6-d2c787096a29 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:11.359665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:11.386711 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c9738-3ba6-4371-90a6-d2c787096a29 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:11.423307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e99f5f-10f3-4b12-9cfc-c75ec8a9573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:11.424215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e99f5f-10f3-4b12-9cfc-c75ec8a9573c None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:11.424502 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e99f5f-10f3-4b12-9cfc-c75ec8a9573c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:11.424786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e99f5f-10f3-4b12-9cfc-c75ec8a9573c None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979/users/e0d5b9edc140499bb612222f6572ae20/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:11.426872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e99f5f-10f3-4b12-9cfc-c75ec8a9573c None admin] RBAC: Authorizing `identity:create_grant(project_id=d76703b19c6c4521a9480e46863c1979, user_id=e0d5b9edc140499bb612222f6572ae20, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:11.428344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e99f5f-10f3-4b12-9cfc-c75ec8a9573c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:11.547587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 54/104] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:23:11 2022] PUT /identity/v3/projects/d76703b19c6c4521a9480e46863c1979/users/e0d5b9edc140499bb612222f6572ae20/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:11.590911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6e659-4979-4bca-8b3a-7b9e8c76a918 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:11.591419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6e659-4979-4bca-8b3a-7b9e8c76a918 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:11.591491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6e659-4979-4bca-8b3a-7b9e8c76a918 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:11.593019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 51/105] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:11.598680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f3c2-15eb-450c-9697-ba1814b3a8db None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:11.598981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f3c2-15eb-450c-9697-ba1814b3a8db None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:11.599268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f3c2-15eb-450c-9697-ba1814b3a8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:11.622244 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c560f3c2-15eb-450c-9697-ba1814b3a8db None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:11.623522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c560f3c2-15eb-450c-9697-ba1814b3a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:11.713550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 55/106] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:11.737609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6e659-4979-4bca-8b3a-7b9e8c76a918 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:11.837397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30e3a8-021b-4e4d-96fc-8bf0486b20d0 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:11.837716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30e3a8-021b-4e4d-96fc-8bf0486b20d0 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:11.837968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30e3a8-021b-4e4d-96fc-8bf0486b20d0 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:11.839363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 56/107] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:11.844387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329fe3d-f060-4b6b-ba1b-b3785e8eb0a2 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:11.844677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329fe3d-f060-4b6b-ba1b-b3785e8eb0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:11.844961 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329fe3d-f060-4b6b-ba1b-b3785e8eb0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:11.875375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e329fe3d-f060-4b6b-ba1b-b3785e8eb0a2 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:11.876893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e329fe3d-f060-4b6b-ba1b-b3785e8eb0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:11.898399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6e659-4979-4bca-8b3a-7b9e8c76a918 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:11.967874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abe293-3b4c-4a3c-b532-d1bad18279e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:11.968688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe293-3b4c-4a3c-b532-d1bad18279e2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:11.968977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe293-3b4c-4a3c-b532-d1bad18279e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:11.969287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe293-3b4c-4a3c-b532-d1bad18279e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:11.990986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abe293-3b4c-4a3c-b532-d1bad18279e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:12.001424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c2abe293-3b4c-4a3c-b532-d1bad18279e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:12.004806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:12.005708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:12.006459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:12.008476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 52/108] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:12.015433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abe293-3b4c-4a3c-b532-d1bad18279e2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:12.021519 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 57/109] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:12.022978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:12.057048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329fe3d-f060-4b6b-ba1b-b3785e8eb0a2 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:12.207819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abe293-3b4c-4a3c-b532-d1bad18279e2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:12.234953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329fe3d-f060-4b6b-ba1b-b3785e8eb0a2 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:12.246989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a483ba-5346-490f-96d7-85fda51dff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:12.248041 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a483ba-5346-490f-96d7-85fda51dff7b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.248472 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a483ba-5346-490f-96d7-85fda51dff7b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.248892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a483ba-5346-490f-96d7-85fda51dff7b None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.249959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a483ba-5346-490f-96d7-85fda51dff7b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:12.250510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a483ba-5346-490f-96d7-85fda51dff7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:12.252156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a483ba-5346-490f-96d7-85fda51dff7b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:12.261676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 53/110] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:12.263496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8431b-0728-49d0-ba76-e82a42a0ae0f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.263981 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8431b-0728-49d0-ba76-e82a42a0ae0f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.264426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8431b-0728-49d0-ba76-e82a42a0ae0f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.266207 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 54/111] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:12.267628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f87c45-fafc-483f-8326-dfe9765bc1a6 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.268060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f87c45-fafc-483f-8326-dfe9765bc1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.268482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f87c45-fafc-483f-8326-dfe9765bc1a6 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.269919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 55/112] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:12.270889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f87c45-fafc-483f-8326-dfe9765bc1a6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:12.275255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216bde3-1315-49bc-801e-dae88906efdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:12.276067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216bde3-1315-49bc-801e-dae88906efdc None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.276347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216bde3-1315-49bc-801e-dae88906efdc None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.276620 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216bde3-1315-49bc-801e-dae88906efdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.277830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216bde3-1315-49bc-801e-dae88906efdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:12.279311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216bde3-1315-49bc-801e-dae88906efdc None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:12.280966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 58/113] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:12.282237 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e21cb3-7c24-4ae4-9b74-26b7b68280af None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.282530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e21cb3-7c24-4ae4-9b74-26b7b68280af None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.282807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e21cb3-7c24-4ae4-9b74-26b7b68280af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.303959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-12e21cb3-7c24-4ae4-9b74-26b7b68280af None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:12.305784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-12e21cb3-7c24-4ae4-9b74-26b7b68280af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:12.357547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:12.383531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:12.399725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 59/114] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:12.401179 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac272ee6-2f0f-48b7-83fb-9ed2c54a02a5 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.401359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac272ee6-2f0f-48b7-83fb-9ed2c54a02a5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.401722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac272ee6-2f0f-48b7-83fb-9ed2c54a02a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.409488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f87c45-fafc-483f-8326-dfe9765bc1a6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:12.421742 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ac272ee6-2f0f-48b7-83fb-9ed2c54a02a5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:12.422963 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ac272ee6-2f0f-48b7-83fb-9ed2c54a02a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:12.448173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f5cbc-3eeb-4a5f-8a1c-fd2ae2b5e754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:12.449310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f5cbc-3eeb-4a5f-8a1c-fd2ae2b5e754 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.449747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f5cbc-3eeb-4a5f-8a1c-fd2ae2b5e754 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.450017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f5cbc-3eeb-4a5f-8a1c-fd2ae2b5e754 None admin] PATH_INFO: `/v3/users/7fd5e6c20d9f4c02b862ed90c707d937` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.465108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f5cbc-3eeb-4a5f-8a1c-fd2ae2b5e754 None admin] RBAC: Authorizing `identity:get_user(user_id=7fd5e6c20d9f4c02b862ed90c707d937)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:12.467020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f5cbc-3eeb-4a5f-8a1c-fd2ae2b5e754 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:12.473615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 56/115] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:12 2022] GET /identity/v3/users/7fd5e6c20d9f4c02b862ed90c707d937 => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:12.473615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f5cbc-3eeb-4a5f-8a1c-fd2ae2b5e754 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:12.493621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 60/116] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:12.502752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7da71-51b3-4051-a636-48888cfdd1d3 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.503054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7da71-51b3-4051-a636-48888cfdd1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.503340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7da71-51b3-4051-a636-48888cfdd1d3 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.509495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 61/117] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:12.509495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7da71-51b3-4051-a636-48888cfdd1d3 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:12.588996 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f5cbc-3eeb-4a5f-8a1c-fd2ae2b5e754 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:12.612227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:12.623551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7da71-51b3-4051-a636-48888cfdd1d3 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:12.625009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c19be-7aa2-4837-99a9-38182d89bfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:12.625867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c19be-7aa2-4837-99a9-38182d89bfaa None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.626147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c19be-7aa2-4837-99a9-38182d89bfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.626417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c19be-7aa2-4837-99a9-38182d89bfaa None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.627128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c19be-7aa2-4837-99a9-38182d89bfaa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:12.628570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c19be-7aa2-4837-99a9-38182d89bfaa None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:12.647394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:12.658866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048293a-c0f0-46be-b48d-115d5d26ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:12.659690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048293a-c0f0-46be-b48d-115d5d26ba5e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.659974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048293a-c0f0-46be-b48d-115d5d26ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.660248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048293a-c0f0-46be-b48d-115d5d26ba5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.670317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048293a-c0f0-46be-b48d-115d5d26ba5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-2048293a-c0f0-46be-b48d-115d5d26ba5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:12.686773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:12.690278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 62/118] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:12.691831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048293a-c0f0-46be-b48d-115d5d26ba5e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:12.718050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 57/119] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:23:12 2022] POST /identity/v3/projects => generated 302 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:12.725486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79644b9a-9e62-4300-97a6-3a38dacf3e3f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.725486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79644b9a-9e62-4300-97a6-3a38dacf3e3f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.725486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79644b9a-9e62-4300-97a6-3a38dacf3e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.743068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-79644b9a-9e62-4300-97a6-3a38dacf3e3f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:12.744301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-79644b9a-9e62-4300-97a6-3a38dacf3e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:12.827871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 58/120] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:12.829506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79644b9a-9e62-4300-97a6-3a38dacf3e3f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:12.838644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048293a-c0f0-46be-b48d-115d5d26ba5e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:12.904340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63195b94-93b1-41c5-8b15-d31fca3811d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:12.905022 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63195b94-93b1-41c5-8b15-d31fca3811d3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:12.905377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63195b94-93b1-41c5-8b15-d31fca3811d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:12.905657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63195b94-93b1-41c5-8b15-d31fca3811d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:12.949349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79644b9a-9e62-4300-97a6-3a38dacf3e3f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:12.971465 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:12.987432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:13.001840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63195b94-93b1-41c5-8b15-d31fca3811d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:13.004146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63195b94-93b1-41c5-8b15-d31fca3811d3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:13.004252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0a28-9175-430e-b233-d1005d301de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:13.005105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0a28-9175-430e-b233-d1005d301de2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:13.005517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0a28-9175-430e-b233-d1005d301de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:13.005854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0a28-9175-430e-b233-d1005d301de2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:13.019701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff0a28-9175-430e-b233-d1005d301de2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c3ff0a28-9175-430e-b233-d1005d301de2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:13.034088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:13.041907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 59/121] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:13.042684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0a28-9175-430e-b233-d1005d301de2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.123175 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 63/122] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:13.124287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63195b94-93b1-41c5-8b15-d31fca3811d3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.172526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0a28-9175-430e-b233-d1005d301de2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:13.208352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278f412-087a-486a-b4fc-39ae0a20ac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:13.209315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278f412-087a-486a-b4fc-39ae0a20ac65 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:13.209718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278f412-087a-486a-b4fc-39ae0a20ac65 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:13.210156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278f412-087a-486a-b4fc-39ae0a20ac65 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:13.211484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278f412-087a-486a-b4fc-39ae0a20ac65 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:13.211902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278f412-087a-486a-b4fc-39ae0a20ac65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:13.213535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278f412-087a-486a-b4fc-39ae0a20ac65 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:13.222974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 60/123] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:13.228779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278f412-087a-486a-b4fc-39ae0a20ac65 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.241540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:13.258106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63195b94-93b1-41c5-8b15-d31fca3811d3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:13.261566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:13.295620 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abd502-5661-4ea0-8030-671ac80dccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:13.296594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abd502-5661-4ea0-8030-671ac80dccba None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:13.296999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abd502-5661-4ea0-8030-671ac80dccba None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:13.297419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abd502-5661-4ea0-8030-671ac80dccba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:13.308636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abd502-5661-4ea0-8030-671ac80dccba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-d6abd502-5661-4ea0-8030-671ac80dccba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:13.319469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:13.320487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 64/124] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:13.326095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abd502-5661-4ea0-8030-671ac80dccba None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.357126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278f412-087a-486a-b4fc-39ae0a20ac65 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:13.401966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffa457-241d-42e6-a21e-b89d1fa40492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:13.402884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffa457-241d-42e6-a21e-b89d1fa40492 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:13.403338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffa457-241d-42e6-a21e-b89d1fa40492 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:13.403753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffa457-241d-42e6-a21e-b89d1fa40492 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:13.404651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffa457-241d-42e6-a21e-b89d1fa40492 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:13.405119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffa457-241d-42e6-a21e-b89d1fa40492 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:13.406753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffa457-241d-42e6-a21e-b89d1fa40492 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:13.418697 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 61/125] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:13.419754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffa457-241d-42e6-a21e-b89d1fa40492 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.438734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abd502-5661-4ea0-8030-671ac80dccba None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:13.486597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e0e49-ada8-4cff-85c5-167b496fbd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:13.487779 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e0e49-ada8-4cff-85c5-167b496fbd8f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:13.488119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e0e49-ada8-4cff-85c5-167b496fbd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:13.488498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e0e49-ada8-4cff-85c5-167b496fbd8f None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:13.489878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e0e49-ada8-4cff-85c5-167b496fbd8f None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:13.491797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e0e49-ada8-4cff-85c5-167b496fbd8f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:13.494499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 65/126] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:13.495691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e0e49-ada8-4cff-85c5-167b496fbd8f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.509717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:13.531444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:13.541740 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffa457-241d-42e6-a21e-b89d1fa40492 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:13.578205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a051-ec54-44ab-9685-e3e5511b5135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:13.579778 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a051-ec54-44ab-9685-e3e5511b5135 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:13.580310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a051-ec54-44ab-9685-e3e5511b5135 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:13.581015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a051-ec54-44ab-9685-e3e5511b5135 None admin] PATH_INFO: `/v3/users/afd88ca575e04b44b3284ec86097cb2e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:13.582488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20a051-ec54-44ab-9685-e3e5511b5135 None admin] RBAC: Authorizing `identity:get_user(user_id=afd88ca575e04b44b3284ec86097cb2e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:13.584083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20a051-ec54-44ab-9685-e3e5511b5135 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:13.586830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 62/127] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/users/afd88ca575e04b44b3284ec86097cb2e => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:13.587792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a051-ec54-44ab-9685-e3e5511b5135 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.616199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e0e49-ada8-4cff-85c5-167b496fbd8f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:13.654702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65763ae5-7479-45b3-aa3b-d3c6592459c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:13.655469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65763ae5-7479-45b3-aa3b-d3c6592459c4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:13.655864 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65763ae5-7479-45b3-aa3b-d3c6592459c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:13.656229 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65763ae5-7479-45b3-aa3b-d3c6592459c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:13.657056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65763ae5-7479-45b3-aa3b-d3c6592459c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:13.657506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65763ae5-7479-45b3-aa3b-d3c6592459c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:13.659058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65763ae5-7479-45b3-aa3b-d3c6592459c4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:13.672906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 66/128] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:13.672906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65763ae5-7479-45b3-aa3b-d3c6592459c4 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.699036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a051-ec54-44ab-9685-e3e5511b5135 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:13.735068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391b35a-d4e6-4e4d-99d5-6a4af8313318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:13.736280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391b35a-d4e6-4e4d-99d5-6a4af8313318 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:13.736791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391b35a-d4e6-4e4d-99d5-6a4af8313318 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:13.737314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391b35a-d4e6-4e4d-99d5-6a4af8313318 None admin] PATH_INFO: `/v3/users/afd88ca575e04b44b3284ec86097cb2e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:13.739531 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391b35a-d4e6-4e4d-99d5-6a4af8313318 None admin] RBAC: Authorizing `identity:get_user(user_id=afd88ca575e04b44b3284ec86097cb2e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:13.741188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391b35a-d4e6-4e4d-99d5-6a4af8313318 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:13.743070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 63/129] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/users/afd88ca575e04b44b3284ec86097cb2e => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:13.744025 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391b35a-d4e6-4e4d-99d5-6a4af8313318 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.752992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:13.783982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65763ae5-7479-45b3-aa3b-d3c6592459c4 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:13.785144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:13.824506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa5390-842d-424e-98c2-ed2a0315dd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:13.825561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa5390-842d-424e-98c2-ed2a0315dd91 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:13.826068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa5390-842d-424e-98c2-ed2a0315dd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:13.826538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa5390-842d-424e-98c2-ed2a0315dd91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:13.827570 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa5390-842d-424e-98c2-ed2a0315dd91 None admin] RBAC: Adding query filter params (role.id=4dc8f8c1b32c4c57805640c209616fde, scope.project.id=d76703b19c6c4521a9480e46863c1979, user.id=7fd5e6c20d9f4c02b862ed90c707d937) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:13.828076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa5390-842d-424e-98c2-ed2a0315dd91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:13.830001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa5390-842d-424e-98c2-ed2a0315dd91 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:13.836573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 67/130] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/role_assignments?user.id=7fd5e6c20d9f4c02b862ed90c707d937&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=4dc8f8c1b32c4c57805640c209616fde => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:13.838287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa5390-842d-424e-98c2-ed2a0315dd91 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.889292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391b35a-d4e6-4e4d-99d5-6a4af8313318 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:13.927230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767871ac-907e-4d7f-bc95-137851493169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:13.928512 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767871ac-907e-4d7f-bc95-137851493169 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:13.929029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767871ac-907e-4d7f-bc95-137851493169 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:13.930174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767871ac-907e-4d7f-bc95-137851493169 None admin] PATH_INFO: `/v3/projects/1198d4d7066b4f8c81523f63f0843bd3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:13.931938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767871ac-907e-4d7f-bc95-137851493169 None admin] RBAC: Authorizing `identity:get_project(project_id=1198d4d7066b4f8c81523f63f0843bd3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:13.933709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767871ac-907e-4d7f-bc95-137851493169 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:13.935785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 64/131] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/projects/1198d4d7066b4f8c81523f63f0843bd3 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:13.936762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767871ac-907e-4d7f-bc95-137851493169 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:13.995930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa5390-842d-424e-98c2-ed2a0315dd91 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:14.040724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3ae17-c9b7-46da-b68d-d82ca7155cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:14.041664 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3ae17-c9b7-46da-b68d-d82ca7155cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:14.041952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3ae17-c9b7-46da-b68d-d82ca7155cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:14.042230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3ae17-c9b7-46da-b68d-d82ca7155cf3 None admin] PATH_INFO: `/v3/users/afd88ca575e04b44b3284ec86097cb2e` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:14.043500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3ae17-c9b7-46da-b68d-d82ca7155cf3 None admin] RBAC: Authorizing `identity:get_user(user_id=afd88ca575e04b44b3284ec86097cb2e)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:14.045026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3ae17-c9b7-46da-b68d-d82ca7155cf3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:14.047702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 68/132] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/users/afd88ca575e04b44b3284ec86097cb2e => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:14.049089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3ae17-c9b7-46da-b68d-d82ca7155cf3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:14.053018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:14.101213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:14.109573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767871ac-907e-4d7f-bc95-137851493169 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:14.147247 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb127e-d6d1-483f-832d-098b3a33f783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:14.148122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb127e-d6d1-483f-832d-098b3a33f783 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:14.148954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb127e-d6d1-483f-832d-098b3a33f783 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:14.149264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb127e-d6d1-483f-832d-098b3a33f783 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:14.150491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb127e-d6d1-483f-832d-098b3a33f783 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:14.151971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb127e-d6d1-483f-832d-098b3a33f783 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:14.153883 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 65/133] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:13 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:14.154843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb127e-d6d1-483f-832d-098b3a33f783 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:14.184825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3ae17-c9b7-46da-b68d-d82ca7155cf3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:14.223370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672972a5-7be4-4d97-886f-3e906d336a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:14.224417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672972a5-7be4-4d97-886f-3e906d336a75 None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:14.224535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672972a5-7be4-4d97-886f-3e906d336a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:14.224820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672972a5-7be4-4d97-886f-3e906d336a75 None admin] PATH_INFO: `/v3/projects/1198d4d7066b4f8c81523f63f0843bd3/users/afd88ca575e04b44b3284ec86097cb2e/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:14.226919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672972a5-7be4-4d97-886f-3e906d336a75 None admin] RBAC: Authorizing `identity:create_grant(project_id=1198d4d7066b4f8c81523f63f0843bd3, user_id=afd88ca575e04b44b3284ec86097cb2e, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:14.228457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672972a5-7be4-4d97-886f-3e906d336a75 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:14.272373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb127e-d6d1-483f-832d-098b3a33f783 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:14.310892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8877f-e725-4bbe-9e70-a7ddb47e082a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:14.312118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8877f-e725-4bbe-9e70-a7ddb47e082a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:14.312547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8877f-e725-4bbe-9e70-a7ddb47e082a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:14.312965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8877f-e725-4bbe-9e70-a7ddb47e082a None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:14.314323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8877f-e725-4bbe-9e70-a7ddb47e082a None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:14.315944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8877f-e725-4bbe-9e70-a7ddb47e082a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:14.318242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 66/134] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:14 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:14.324137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b5e0-8d99-4f7e-8a0e-c4e35d22cb0f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:14.324137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b5e0-8d99-4f7e-8a0e-c4e35d22cb0f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:14.324137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b5e0-8d99-4f7e-8a0e-c4e35d22cb0f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:14.324137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 67/135] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:14.324137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05b5e0-8d99-4f7e-8a0e-c4e35d22cb0f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:14.340100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 69/136] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:23:14 2022] PUT /identity/v3/projects/1198d4d7066b4f8c81523f63f0843bd3/users/afd88ca575e04b44b3284ec86097cb2e/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:14.341449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672972a5-7be4-4d97-886f-3e906d336a75 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:14.381632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:14.427575 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:14.482592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672972a5-7be4-4d97-886f-3e906d336a75 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:14.498911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05b5e0-8d99-4f7e-8a0e-c4e35d22cb0f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:14.526391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ab78d-59b2-436b-94d6-e9df6cde38d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:14.527377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ab78d-59b2-436b-94d6-e9df6cde38d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:14.527670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ab78d-59b2-436b-94d6-e9df6cde38d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:14.527947 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ab78d-59b2-436b-94d6-e9df6cde38d4 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979/users/afd88ca575e04b44b3284ec86097cb2e/roles/41fd0da9b16d41c08992073473d87488` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:14.531077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ab78d-59b2-436b-94d6-e9df6cde38d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d76703b19c6c4521a9480e46863c1979, user_id=afd88ca575e04b44b3284ec86097cb2e, role_id=41fd0da9b16d41c08992073473d87488)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:14.532847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ab78d-59b2-436b-94d6-e9df6cde38d4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:14.550946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ba2a8-eea4-4b89-8964-e11d173595c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:14.551831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ba2a8-eea4-4b89-8964-e11d173595c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:14.556499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ba2a8-eea4-4b89-8964-e11d173595c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:14.556499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ba2a8-eea4-4b89-8964-e11d173595c2 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979/users/afd88ca575e04b44b3284ec86097cb2e/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:14.556499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ba2a8-eea4-4b89-8964-e11d173595c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d76703b19c6c4521a9480e46863c1979, user_id=afd88ca575e04b44b3284ec86097cb2e, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:14.556499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ba2a8-eea4-4b89-8964-e11d173595c2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:14.650328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 70/137] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:23:14 2022] PUT /identity/v3/projects/d76703b19c6c4521a9480e46863c1979/users/afd88ca575e04b44b3284ec86097cb2e/roles/41fd0da9b16d41c08992073473d87488 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:14.651633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca820f8-de71-42aa-98b3-fae5dc30631c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:14.651923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca820f8-de71-42aa-98b3-fae5dc30631c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:14.652200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca820f8-de71-42aa-98b3-fae5dc30631c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:14.664594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 68/138] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:23:14 2022] PUT /identity/v3/projects/d76703b19c6c4521a9480e46863c1979/users/afd88ca575e04b44b3284ec86097cb2e/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:14.675930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9ca820f8-de71-42aa-98b3-fae5dc30631c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:14.677194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9ca820f8-de71-42aa-98b3-fae5dc30631c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:14.777202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 71/139] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:14.815678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ba2a8-eea4-4b89-8964-e11d173595c2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:14.956815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:14.970796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ba2a8-eea4-4b89-8964-e11d173595c2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:15.009550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3591a8a-d7b3-4b47-9e74-d40534f74ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:15.010356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3591a8a-d7b3-4b47-9e74-d40534f74ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:15.010639 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3591a8a-d7b3-4b47-9e74-d40534f74ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:15.010917 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3591a8a-d7b3-4b47-9e74-d40534f74ae3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:15.027593 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3591a8a-d7b3-4b47-9e74-d40534f74ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f3591a8a-d7b3-4b47-9e74-d40534f74ae3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:15.036790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:15.037663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:15.038430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:15.041417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 69/140] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:23:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:15.047633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca820f8-de71-42aa-98b3-fae5dc30631c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:15.130128 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:15.183451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca820f8-de71-42aa-98b3-fae5dc30631c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:15.227242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13b49f-a9a9-4535-b4a5-149b3473c12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:15.227921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13b49f-a9a9-4535-b4a5-149b3473c12c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:15.228206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13b49f-a9a9-4535-b4a5-149b3473c12c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:15.228484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13b49f-a9a9-4535-b4a5-149b3473c12c None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:15.229278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13b49f-a9a9-4535-b4a5-149b3473c12c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:15.229474 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91871665-bdc5-4d9f-8a37-45527257ad22 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:15.229571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13b49f-a9a9-4535-b4a5-149b3473c12c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:15.230035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91871665-bdc5-4d9f-8a37-45527257ad22 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:15.230523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91871665-bdc5-4d9f-8a37-45527257ad22 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:15.230963 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13b49f-a9a9-4535-b4a5-149b3473c12c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:15.232389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 70/141] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:23:15.239720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 72/142] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:15.241188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f429b9-5ed4-47f7-adaf-2b7c1a716b5e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:15.241468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f429b9-5ed4-47f7-adaf-2b7c1a716b5e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:15.241744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f429b9-5ed4-47f7-adaf-2b7c1a716b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:15.245478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13b49f-a9a9-4535-b4a5-149b3473c12c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:15.266440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a6f429b9-5ed4-47f7-adaf-2b7c1a716b5e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:15.267631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a6f429b9-5ed4-47f7-adaf-2b7c1a716b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:15.443543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13b49f-a9a9-4535-b4a5-149b3473c12c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:15.460768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 71/143] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:15.462427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:15.482544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb466df-d729-426f-a760-90b5fa23a9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:15.483415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb466df-d729-426f-a760-90b5fa23a9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:15.483927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb466df-d729-426f-a760-90b5fa23a9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:15.484074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb466df-d729-426f-a760-90b5fa23a9e4 None admin] PATH_INFO: `/v3/users/e0d5b9edc140499bb612222f6572ae20` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:15.485521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb466df-d729-426f-a760-90b5fa23a9e4 None admin] RBAC: Authorizing `identity:get_user(user_id=e0d5b9edc140499bb612222f6572ae20)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:15.486992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb466df-d729-426f-a760-90b5fa23a9e4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:15.488633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 73/144] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:15 2022] GET /identity/v3/users/e0d5b9edc140499bb612222f6572ae20 => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:15.494953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f429b9-5ed4-47f7-adaf-2b7c1a716b5e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:15.501016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb466df-d729-426f-a760-90b5fa23a9e4 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:15.626814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:15.646736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f429b9-5ed4-47f7-adaf-2b7c1a716b5e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:15.683984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7524-ee94-4c42-97e7-9b26f914d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:15.684644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7524-ee94-4c42-97e7-9b26f914d24f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:15.684984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7524-ee94-4c42-97e7-9b26f914d24f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:15.685293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7524-ee94-4c42-97e7-9b26f914d24f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:15.725557 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb466df-d729-426f-a760-90b5fa23a9e4 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:15.758851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f7524-ee94-4c42-97e7-9b26f914d24f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:15.760317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f7524-ee94-4c42-97e7-9b26f914d24f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:15.786847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7f51a-d224-452e-8395-822401c6be67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:15.787653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7f51a-d224-452e-8395-822401c6be67 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:15.787936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7f51a-d224-452e-8395-822401c6be67 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:15.788212 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7f51a-d224-452e-8395-822401c6be67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:15.793890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7f51a-d224-452e-8395-822401c6be67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:15.795387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7f51a-d224-452e-8395-822401c6be67 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:15.797258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 74/145] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:15.803243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7f51a-d224-452e-8395-822401c6be67 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:15.899772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:15.916440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:15.971156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7f51a-d224-452e-8395-822401c6be67 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:15.975838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 72/146] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:15.982107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7524-ee94-4c42-97e7-9b26f914d24f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:16.018503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea23f1-b21a-4f93-a130-b24230f601ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:16.019193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea23f1-b21a-4f93-a130-b24230f601ad None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:16.019474 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea23f1-b21a-4f93-a130-b24230f601ad None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:16.019749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea23f1-b21a-4f93-a130-b24230f601ad None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:16.020492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea23f1-b21a-4f93-a130-b24230f601ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:16.022032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea23f1-b21a-4f93-a130-b24230f601ad None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:16.145127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7524-ee94-4c42-97e7-9b26f914d24f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:16.169724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 75/147] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:15 2022] POST /identity/v3/users => generated 283 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:23:16.207137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26952296-dd90-4ad2-afed-91232ed0bf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:16.208053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26952296-dd90-4ad2-afed-91232ed0bf3f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:16.208336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26952296-dd90-4ad2-afed-91232ed0bf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:16.208609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26952296-dd90-4ad2-afed-91232ed0bf3f None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:16.210014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26952296-dd90-4ad2-afed-91232ed0bf3f None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:16.211558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26952296-dd90-4ad2-afed-91232ed0bf3f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:16.213391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 73/148] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:15 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:16.219822 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea23f1-b21a-4f93-a130-b24230f601ad None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:16.359792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea23f1-b21a-4f93-a130-b24230f601ad None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:16.374030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:16.405939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0c71a-d382-4bfc-80e5-4ea1b65e6680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:16.406622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0c71a-d382-4bfc-80e5-4ea1b65e6680 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:16.406903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0c71a-d382-4bfc-80e5-4ea1b65e6680 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:16.407182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0c71a-d382-4bfc-80e5-4ea1b65e6680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:16.408043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0c71a-d382-4bfc-80e5-4ea1b65e6680 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=d76703b19c6c4521a9480e46863c1979, user.id=e0d5b9edc140499bb612222f6572ae20) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:16.408367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0c71a-d382-4bfc-80e5-4ea1b65e6680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:16.415039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0c71a-d382-4bfc-80e5-4ea1b65e6680 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:16.422577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 76/149] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:16 2022] GET /identity/v3/role_assignments?user.id=e0d5b9edc140499bb612222f6572ae20&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:16.677236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b634b-639e-4fce-93c6-01f31a078774 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:16.677571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b634b-639e-4fce-93c6-01f31a078774 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:16.677774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b634b-639e-4fce-93c6-01f31a078774 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:16.679268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 74/150] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:16.686415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6778627-f837-489f-bd6d-e844ce8ada48 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:16.686892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6778627-f837-489f-bd6d-e844ce8ada48 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:16.686962 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6778627-f837-489f-bd6d-e844ce8ada48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:16.714676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-d6778627-f837-489f-bd6d-e844ce8ada48 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:16.715895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d6778627-f837-489f-bd6d-e844ce8ada48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:16.814675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 77/151] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:16.845604 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b634b-639e-4fce-93c6-01f31a078774 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:16.923085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:16.996045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b634b-639e-4fce-93c6-01f31a078774 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:17.037542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c27aa7-ce82-4b4b-bfa3-c145b4d8d43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:17.038352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c27aa7-ce82-4b4b-bfa3-c145b4d8d43e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.038649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c27aa7-ce82-4b4b-bfa3-c145b4d8d43e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.038908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c27aa7-ce82-4b4b-bfa3-c145b4d8d43e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.052916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c27aa7-ce82-4b4b-bfa3-c145b4d8d43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-90c27aa7-ce82-4b4b-bfa3-c145b4d8d43e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:17.061647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:17.062654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:17.063872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 75/152] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:17.072348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6778627-f837-489f-bd6d-e844ce8ada48 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:17.210084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6778627-f837-489f-bd6d-e844ce8ada48 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:17.212191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:17.250972 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a9767-cc33-46bb-aa04-6227e81a65ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:17.251569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a9767-cc33-46bb-aa04-6227e81a65ad None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.251855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a9767-cc33-46bb-aa04-6227e81a65ad None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.252127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a9767-cc33-46bb-aa04-6227e81a65ad None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.252886 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a9767-cc33-46bb-aa04-6227e81a65ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:17.253221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a9767-cc33-46bb-aa04-6227e81a65ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:17.254609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a9767-cc33-46bb-aa04-6227e81a65ad None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:17.264598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 78/153] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:17.269995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c27aa7-ce82-4b4b-bfa3-c145b4d8d43e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:17.426226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c27aa7-ce82-4b4b-bfa3-c145b4d8d43e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:17.439050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:17.472135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49d990-a67d-477c-a2fb-836832bb4c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:17.473018 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49d990-a67d-477c-a2fb-836832bb4c68 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.473326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49d990-a67d-477c-a2fb-836832bb4c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.473605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49d990-a67d-477c-a2fb-836832bb4c68 None admin] PATH_INFO: `/v3/users/7fd5e6c20d9f4c02b862ed90c707d937` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.477922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49d990-a67d-477c-a2fb-836832bb4c68 None admin] RBAC: Authorizing `identity:get_user(user_id=7fd5e6c20d9f4c02b862ed90c707d937)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:17.479489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49d990-a67d-477c-a2fb-836832bb4c68 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:17.481652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 76/154] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:17 2022] GET /identity/v3/users/7fd5e6c20d9f4c02b862ed90c707d937 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:17.482553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3cbe2-7905-4269-931b-1020babb9707 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.482812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3cbe2-7905-4269-931b-1020babb9707 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.483091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3cbe2-7905-4269-931b-1020babb9707 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.484694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 79/155] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:17.486988 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49d990-a67d-477c-a2fb-836832bb4c68 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:17.493551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1691f3-8d2c-4637-b356-1a718acacd2a None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.493835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1691f3-8d2c-4637-b356-1a718acacd2a None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.494111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1691f3-8d2c-4637-b356-1a718acacd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.526339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ac1691f3-8d2c-4637-b356-1a718acacd2a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:17.527650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ac1691f3-8d2c-4637-b356-1a718acacd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:17.626863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 80/156] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:17.628547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f3e72-6693-4446-98f2-31568e15f41e None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.629013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f3e72-6693-4446-98f2-31568e15f41e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.629115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49d990-a67d-477c-a2fb-836832bb4c68 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:17.631770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f3e72-6693-4446-98f2-31568e15f41e None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.633581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 81/157] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:17.641648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bc345-ee92-44b4-801b-3b9b3175621c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.642185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bc345-ee92-44b4-801b-3b9b3175621c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.642629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bc345-ee92-44b4-801b-3b9b3175621c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.665736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-047bc345-ee92-44b4-801b-3b9b3175621c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:17.667238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-047bc345-ee92-44b4-801b-3b9b3175621c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:17.689085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb729750-566e-40ce-8f36-feea272d61c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:17.689982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb729750-566e-40ce-8f36-feea272d61c2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.690263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb729750-566e-40ce-8f36-feea272d61c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.690532 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb729750-566e-40ce-8f36-feea272d61c2 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.692534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb729750-566e-40ce-8f36-feea272d61c2 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:17.694113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb729750-566e-40ce-8f36-feea272d61c2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:17.697313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 77/158] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:17 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:17.698408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb729750-566e-40ce-8f36-feea272d61c2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:17.760114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 82/159] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:17.760967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bc345-ee92-44b4-801b-3b9b3175621c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:17.765524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:17.794012 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:17.841133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb729750-566e-40ce-8f36-feea272d61c2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:17.878340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bc345-ee92-44b4-801b-3b9b3175621c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:17.892905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d3a4d-560f-4d03-a8af-b3bc77b6a2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:17.894013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d3a4d-560f-4d03-a8af-b3bc77b6a2ac None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.894821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d3a4d-560f-4d03-a8af-b3bc77b6a2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.895107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d3a4d-560f-4d03-a8af-b3bc77b6a2ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.909624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d3a4d-560f-4d03-a8af-b3bc77b6a2ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:17.917656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277953fb-c8a2-4236-9c80-ac5375eebcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:17.918544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277953fb-c8a2-4236-9c80-ac5375eebcff None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.918826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277953fb-c8a2-4236-9c80-ac5375eebcff None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4b5d3a4d-560f-4d03-a8af-b3bc77b6a2ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:17.918936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:17.919751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:17.920520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:17.921110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277953fb-c8a2-4236-9c80-ac5375eebcff None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979/users/7fd5e6c20d9f4c02b862ed90c707d937/roles/4dc8f8c1b32c4c57805640c209616fde` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.921224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 78/160] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:17.921295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277953fb-c8a2-4236-9c80-ac5375eebcff None admin] RBAC: Authorizing `identity:create_grant(project_id=d76703b19c6c4521a9480e46863c1979, user_id=7fd5e6c20d9f4c02b862ed90c707d937, role_id=4dc8f8c1b32c4c57805640c209616fde)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:17.924332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e366262-4d38-406c-8a12-e46416a714f3 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.924332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e366262-4d38-406c-8a12-e46416a714f3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.924332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e366262-4d38-406c-8a12-e46416a714f3 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.924332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 79/161] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:17.925184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e366262-4d38-406c-8a12-e46416a714f3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:17.926957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277953fb-c8a2-4236-9c80-ac5375eebcff None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:18.041212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e366262-4d38-406c-8a12-e46416a714f3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:18.043737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:18.078779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c76dec-a8a8-4099-a2ec-0762771be20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:18.079548 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c76dec-a8a8-4099-a2ec-0762771be20d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:18.079825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c76dec-a8a8-4099-a2ec-0762771be20d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:18.080190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c76dec-a8a8-4099-a2ec-0762771be20d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:18.092169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c76dec-a8a8-4099-a2ec-0762771be20d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-59c76dec-a8a8-4099-a2ec-0762771be20d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:18.101557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:18.103445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 80/162] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:18.104806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7bdf9-caaa-44d5-b887-ca365bbcc930 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:18.115000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7bdf9-caaa-44d5-b887-ca365bbcc930 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:18.115166 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 83/163] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:23:17 2022] PUT /identity/v3/projects/d76703b19c6c4521a9480e46863c1979/users/7fd5e6c20d9f4c02b862ed90c707d937/roles/4dc8f8c1b32c4c57805640c209616fde => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:18.115322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7bdf9-caaa-44d5-b887-ca365bbcc930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:18.115463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277953fb-c8a2-4236-9c80-ac5375eebcff None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:18.139152 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-86b7bdf9-caaa-44d5-b887-ca365bbcc930 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:18.140380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-86b7bdf9-caaa-44d5-b887-ca365bbcc930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:18.276801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:18.278435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 81/164] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:18.279378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7bdf9-caaa-44d5-b887-ca365bbcc930 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:18.307039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277953fb-c8a2-4236-9c80-ac5375eebcff None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:18.366853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640b4b6-d974-4396-bb0c-c9a7ffed4d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:18.367456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640b4b6-d974-4396-bb0c-c9a7ffed4d53 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:18.367735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640b4b6-d974-4396-bb0c-c9a7ffed4d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:18.368007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640b4b6-d974-4396-bb0c-c9a7ffed4d53 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:18.368773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640b4b6-d974-4396-bb0c-c9a7ffed4d53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:18.369136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640b4b6-d974-4396-bb0c-c9a7ffed4d53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:18.370550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640b4b6-d974-4396-bb0c-c9a7ffed4d53 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:18.389553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 84/165] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:18.390564 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640b4b6-d974-4396-bb0c-c9a7ffed4d53 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:18.463619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:18.479491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7bdf9-caaa-44d5-b887-ca365bbcc930 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:18.559440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042df67-5c5c-4b97-a15b-c180424ac1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:18.560130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042df67-5c5c-4b97-a15b-c180424ac1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:18.560413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042df67-5c5c-4b97-a15b-c180424ac1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:18.560686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042df67-5c5c-4b97-a15b-c180424ac1c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:18.561726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042df67-5c5c-4b97-a15b-c180424ac1c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:18.562031 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042df67-5c5c-4b97-a15b-c180424ac1c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:18.563416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042df67-5c5c-4b97-a15b-c180424ac1c4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:18.584763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 82/166] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:18.595249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042df67-5c5c-4b97-a15b-c180424ac1c4 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:18.611434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640b4b6-d974-4396-bb0c-c9a7ffed4d53 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:18.671060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aef0bc-2ac2-44dd-91bc-d2b700acdd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:18.671961 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aef0bc-2ac2-44dd-91bc-d2b700acdd07 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:18.672248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aef0bc-2ac2-44dd-91bc-d2b700acdd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:18.672522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aef0bc-2ac2-44dd-91bc-d2b700acdd07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:18.693658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aef0bc-2ac2-44dd-91bc-d2b700acdd07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-95aef0bc-2ac2-44dd-91bc-d2b700acdd07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:18.708534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:18.709492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:18.710277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:18.711438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 85/167] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:18.713711 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aef0bc-2ac2-44dd-91bc-d2b700acdd07 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:18.729493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:18.853436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042df67-5c5c-4b97-a15b-c180424ac1c4 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:18.870206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aef0bc-2ac2-44dd-91bc-d2b700acdd07 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:18.919724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404057a-7bb6-497e-9e1f-7e0c68d4ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:18.920599 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404057a-7bb6-497e-9e1f-7e0c68d4ae6a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:18.920877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404057a-7bb6-497e-9e1f-7e0c68d4ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:18.921214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404057a-7bb6-497e-9e1f-7e0c68d4ae6a None admin] PATH_INFO: `/v3/users/afd88ca575e04b44b3284ec86097cb2e` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:18.925880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404057a-7bb6-497e-9e1f-7e0c68d4ae6a None admin] RBAC: Authorizing `identity:get_user(user_id=afd88ca575e04b44b3284ec86097cb2e)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:18.927358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404057a-7bb6-497e-9e1f-7e0c68d4ae6a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:18.929496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 86/168] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:18 2022] GET /identity/v3/users/afd88ca575e04b44b3284ec86097cb2e => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:18.930370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404057a-7bb6-497e-9e1f-7e0c68d4ae6a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:18.948463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8d4e8-d907-49c9-8569-d369775e82bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:18.950058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d4e8-d907-49c9-8569-d369775e82bb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:18.950573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d4e8-d907-49c9-8569-d369775e82bb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:18.951070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d4e8-d907-49c9-8569-d369775e82bb None admin] PATH_INFO: `/v3/users/afd88ca575e04b44b3284ec86097cb2e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:18.959932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8d4e8-d907-49c9-8569-d369775e82bb None admin] RBAC: Authorizing `identity:get_user(user_id=afd88ca575e04b44b3284ec86097cb2e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:18.962376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8d4e8-d907-49c9-8569-d369775e82bb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:18.964447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 83/169] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:18 2022] GET /identity/v3/users/afd88ca575e04b44b3284ec86097cb2e => generated 275 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:18.965302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8d4e8-d907-49c9-8569-d369775e82bb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:19.045520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:19.060465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:19.101039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404057a-7bb6-497e-9e1f-7e0c68d4ae6a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:19.162003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66869c67-d098-4645-8b9b-22ad0b798589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:19.163113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66869c67-d098-4645-8b9b-22ad0b798589 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:19.164171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66869c67-d098-4645-8b9b-22ad0b798589 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:19.164674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66869c67-d098-4645-8b9b-22ad0b798589 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:19.165681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66869c67-d098-4645-8b9b-22ad0b798589 None admin] RBAC: Adding query filter params (name=member) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:19.173250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66869c67-d098-4645-8b9b-22ad0b798589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:19.175070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66869c67-d098-4645-8b9b-22ad0b798589 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:19.188715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 87/170] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:19.189948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66869c67-d098-4645-8b9b-22ad0b798589 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:19.227871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8d4e8-d907-49c9-8569-d369775e82bb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:19.318158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66869c67-d098-4645-8b9b-22ad0b798589 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:19.325922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fce4c-8c77-4e69-83be-65f8f82b36d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:19.326608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fce4c-8c77-4e69-83be-65f8f82b36d1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:19.326896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fce4c-8c77-4e69-83be-65f8f82b36d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:19.327167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fce4c-8c77-4e69-83be-65f8f82b36d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:19.374748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078be939-8157-4cef-9809-8fedeb805782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:19.377421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078be939-8157-4cef-9809-8fedeb805782 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:19.377421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078be939-8157-4cef-9809-8fedeb805782 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:19.377421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078be939-8157-4cef-9809-8fedeb805782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:19.402654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:19.481242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fce4c-8c77-4e69-83be-65f8f82b36d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:19.482628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fce4c-8c77-4e69-83be-65f8f82b36d1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:19.515234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078be939-8157-4cef-9809-8fedeb805782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:19.516702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078be939-8157-4cef-9809-8fedeb805782 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:19.553601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:19.644262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 84/171] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:19.645123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fce4c-8c77-4e69-83be-65f8f82b36d1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:19.685572 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 88/172] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:19.686464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078be939-8157-4cef-9809-8fedeb805782 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:19.785631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:19.797088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fce4c-8c77-4e69-83be-65f8f82b36d1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:19.818904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078be939-8157-4cef-9809-8fedeb805782 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:19.837620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:19.857886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9130cb-db42-4370-86ef-b9f63da99a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:19.865271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9130cb-db42-4370-86ef-b9f63da99a82 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:19.865271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9130cb-db42-4370-86ef-b9f63da99a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:19.865271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9130cb-db42-4370-86ef-b9f63da99a82 None admin] PATH_INFO: `/v3/users/afd88ca575e04b44b3284ec86097cb2e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:19.865271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9130cb-db42-4370-86ef-b9f63da99a82 None admin] RBAC: Authorizing `identity:get_user(user_id=afd88ca575e04b44b3284ec86097cb2e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:19.865271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9130cb-db42-4370-86ef-b9f63da99a82 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:19.865271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 85/173] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:19 2022] GET /identity/v3/users/afd88ca575e04b44b3284ec86097cb2e => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:19.867478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293eede-a8c7-4ebd-befc-41316de58e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:19.868532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293eede-a8c7-4ebd-befc-41316de58e0f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:19.868964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293eede-a8c7-4ebd-befc-41316de58e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:19.869749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293eede-a8c7-4ebd-befc-41316de58e0f None admin] PATH_INFO: `/v3/projects/1198d4d7066b4f8c81523f63f0843bd3` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:19.871314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293eede-a8c7-4ebd-befc-41316de58e0f None admin] RBAC: Authorizing `identity:get_project(project_id=1198d4d7066b4f8c81523f63f0843bd3)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:19.873386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293eede-a8c7-4ebd-befc-41316de58e0f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:19.873516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9130cb-db42-4370-86ef-b9f63da99a82 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:19.875376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 89/174] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:19 2022] GET /identity/v3/projects/1198d4d7066b4f8c81523f63f0843bd3 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:19.876840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5b60-8a3c-487c-a361-a309d09aaad6 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:19.877302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5b60-8a3c-487c-a361-a309d09aaad6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:19.877726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5b60-8a3c-487c-a361-a309d09aaad6 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:19.882974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 90/175] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:19.883956 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c5b60-8a3c-487c-a361-a309d09aaad6 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:19.991268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c5b60-8a3c-487c-a361-a309d09aaad6 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:20.002299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:20.028227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec129ec-d8c0-4edc-a5ff-6ec1a0adeecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:20.029109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec129ec-d8c0-4edc-a5ff-6ec1a0adeecc None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.029630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec129ec-d8c0-4edc-a5ff-6ec1a0adeecc None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.030061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec129ec-d8c0-4edc-a5ff-6ec1a0adeecc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.075470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9130cb-db42-4370-86ef-b9f63da99a82 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:20.100781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec129ec-d8c0-4edc-a5ff-6ec1a0adeecc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:20.102604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec129ec-d8c0-4edc-a5ff-6ec1a0adeecc None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:20.146419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977129c9-b034-47c6-9187-410f5f3e5c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:20.147298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977129c9-b034-47c6-9187-410f5f3e5c3d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.147582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977129c9-b034-47c6-9187-410f5f3e5c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.147875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977129c9-b034-47c6-9187-410f5f3e5c3d None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.153886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977129c9-b034-47c6-9187-410f5f3e5c3d None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:20.155340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977129c9-b034-47c6-9187-410f5f3e5c3d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:20.157053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 86/176] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:19 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:20.157839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977129c9-b034-47c6-9187-410f5f3e5c3d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:20.209269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 91/177] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:20.210885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c3d3a-6c89-4f7e-9b4f-b63cd537e6b5 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.211325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c3d3a-6c89-4f7e-9b4f-b63cd537e6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.211744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c3d3a-6c89-4f7e-9b4f-b63cd537e6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.225554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:20.229879 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-188c3d3a-6c89-4f7e-9b4f-b63cd537e6b5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:20.231224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-188c3d3a-6c89-4f7e-9b4f-b63cd537e6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:20.265581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:20.314048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977129c9-b034-47c6-9187-410f5f3e5c3d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:20.324422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 92/178] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:20.325113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c3d3a-6c89-4f7e-9b4f-b63cd537e6b5 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:20.402877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f4228-e0ac-4a9c-b315-63ad5f0c84d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:20.403466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f4228-e0ac-4a9c-b315-63ad5f0c84d6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.403745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f4228-e0ac-4a9c-b315-63ad5f0c84d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.404022 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f4228-e0ac-4a9c-b315-63ad5f0c84d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.404870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f4228-e0ac-4a9c-b315-63ad5f0c84d6 None admin] RBAC: Adding query filter params (role.id=df5932e731294f7498a6983a730628bd, scope.project.id=1198d4d7066b4f8c81523f63f0843bd3, user.id=afd88ca575e04b44b3284ec86097cb2e) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:20.405196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f4228-e0ac-4a9c-b315-63ad5f0c84d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:20.406651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f4228-e0ac-4a9c-b315-63ad5f0c84d6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:20.419423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 87/179] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:20 2022] GET /identity/v3/role_assignments?user.id=afd88ca575e04b44b3284ec86097cb2e&scope.project.id=1198d4d7066b4f8c81523f63f0843bd3&role.id=df5932e731294f7498a6983a730628bd => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:20.420405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f4228-e0ac-4a9c-b315-63ad5f0c84d6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:20.470756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c3d3a-6c89-4f7e-9b4f-b63cd537e6b5 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:20.547346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51109dd0-4ecd-4d18-ba5d-129567177632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:20.550033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51109dd0-4ecd-4d18-ba5d-129567177632 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.550405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51109dd0-4ecd-4d18-ba5d-129567177632 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.550697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51109dd0-4ecd-4d18-ba5d-129567177632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.551569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51109dd0-4ecd-4d18-ba5d-129567177632 None admin] RBAC: Adding query filter params (role.id=41fd0da9b16d41c08992073473d87488, scope.project.id=d76703b19c6c4521a9480e46863c1979, user.id=afd88ca575e04b44b3284ec86097cb2e) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:20.551882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51109dd0-4ecd-4d18-ba5d-129567177632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:20.553429 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51109dd0-4ecd-4d18-ba5d-129567177632 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:20.556016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:20.571769 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 93/180] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:20 2022] GET /identity/v3/role_assignments?user.id=afd88ca575e04b44b3284ec86097cb2e&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=41fd0da9b16d41c08992073473d87488 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:20.572903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51109dd0-4ecd-4d18-ba5d-129567177632 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:20.645703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f4228-e0ac-4a9c-b315-63ad5f0c84d6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:20.693902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:20.714839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3afa8-1873-4de5-bde5-7516b8e512c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:20.715838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3afa8-1873-4de5-bde5-7516b8e512c9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.716180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3afa8-1873-4de5-bde5-7516b8e512c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.717062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3afa8-1873-4de5-bde5-7516b8e512c9 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.718534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3afa8-1873-4de5-bde5-7516b8e512c9 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:20.720411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3afa8-1873-4de5-bde5-7516b8e512c9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:20.722780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 88/181] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:20 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:20.724560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1631909-5957-4edd-aaf8-6e8fe82b9f1d None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.724843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1631909-5957-4edd-aaf8-6e8fe82b9f1d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.725115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1631909-5957-4edd-aaf8-6e8fe82b9f1d None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.727330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 89/182] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:20.728861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341cc64-aa8c-4d9e-bc0f-858616e89162 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.729146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341cc64-aa8c-4d9e-bc0f-858616e89162 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.729455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341cc64-aa8c-4d9e-bc0f-858616e89162 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.730769 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 90/183] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:20.731568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341cc64-aa8c-4d9e-bc0f-858616e89162 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:20.767836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51109dd0-4ecd-4d18-ba5d-129567177632 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:20.822439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09dc93-1498-41df-9ebf-97ba977f40fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:20.823534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09dc93-1498-41df-9ebf-97ba977f40fe None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.823957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09dc93-1498-41df-9ebf-97ba977f40fe None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.824935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09dc93-1498-41df-9ebf-97ba977f40fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.857200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09dc93-1498-41df-9ebf-97ba977f40fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-4a09dc93-1498-41df-9ebf-97ba977f40fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:20.870855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:20.874838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:20.875772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 94/184] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:23:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:20.878197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dfd91-222a-43aa-9aa5-6418c8d7a7a3 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.878884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dfd91-222a-43aa-9aa5-6418c8d7a7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.879327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dfd91-222a-43aa-9aa5-6418c8d7a7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.908004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341cc64-aa8c-4d9e-bc0f-858616e89162 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:21.018269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c91dfd91-222a-43aa-9aa5-6418c8d7a7a3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:21.019485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c91dfd91-222a-43aa-9aa5-6418c8d7a7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:21.041450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f62a6-c0bb-4e0a-9c45-99113502e0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:21.042211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f62a6-c0bb-4e0a-9c45-99113502e0ac None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.042574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f62a6-c0bb-4e0a-9c45-99113502e0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.042897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f62a6-c0bb-4e0a-9c45-99113502e0ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.043805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f62a6-c0bb-4e0a-9c45-99113502e0ac None admin] RBAC: Adding query filter params (role.id=df5932e731294f7498a6983a730628bd, scope.project.id=d76703b19c6c4521a9480e46863c1979, user.id=afd88ca575e04b44b3284ec86097cb2e) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:21.044206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f62a6-c0bb-4e0a-9c45-99113502e0ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:21.045758 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f62a6-c0bb-4e0a-9c45-99113502e0ac None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:21.065421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 91/185] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:20 2022] GET /identity/v3/role_assignments?user.id=afd88ca575e04b44b3284ec86097cb2e&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=df5932e731294f7498a6983a730628bd => generated 608 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:21.066896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1961f98-c05e-483d-93ac-688d8bd909ac None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.067188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1961f98-c05e-483d-93ac-688d8bd909ac None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.067489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1961f98-c05e-483d-93ac-688d8bd909ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.082088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:21.101978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c1961f98-c05e-483d-93ac-688d8bd909ac None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:21.113467 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c1961f98-c05e-483d-93ac-688d8bd909ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:21.113691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 95/186] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:21.113691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dfd91-222a-43aa-9aa5-6418c8d7a7a3 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:21.182370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:21.201053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 92/187] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:21.209429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdeb47-9cc0-4687-b922-4ff9aba5c534 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.209429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdeb47-9cc0-4687-b922-4ff9aba5c534 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.209429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdeb47-9cc0-4687-b922-4ff9aba5c534 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.209652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 93/188] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:21.213177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdeb47-9cc0-4687-b922-4ff9aba5c534 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:21.243776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dfd91-222a-43aa-9aa5-6418c8d7a7a3 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:21.286777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d914ff-8ba2-44d8-827e-d5ff028c9978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:21.287376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d914ff-8ba2-44d8-827e-d5ff028c9978 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.287659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d914ff-8ba2-44d8-827e-d5ff028c9978 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.291330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d914ff-8ba2-44d8-827e-d5ff028c9978 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.291330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d914ff-8ba2-44d8-827e-d5ff028c9978 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:21.291330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d914ff-8ba2-44d8-827e-d5ff028c9978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:21.291330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d914ff-8ba2-44d8-827e-d5ff028c9978 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:21.302904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 96/189] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:21.304896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb71cbc-051c-49a0-b4ad-4f6429affa3f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.305190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb71cbc-051c-49a0-b4ad-4f6429affa3f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.305471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb71cbc-051c-49a0-b4ad-4f6429affa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.323048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-3bb71cbc-051c-49a0-b4ad-4f6429affa3f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:21.324247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3bb71cbc-051c-49a0-b4ad-4f6429affa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:21.331991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdeb47-9cc0-4687-b922-4ff9aba5c534 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:21.365287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11b63c-4e1b-456e-9512-53cda69c698d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:21.366087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11b63c-4e1b-456e-9512-53cda69c698d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.366367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11b63c-4e1b-456e-9512-53cda69c698d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.366675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11b63c-4e1b-456e-9512-53cda69c698d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.371967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:21.377800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11b63c-4e1b-456e-9512-53cda69c698d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-7a11b63c-4e1b-456e-9512-53cda69c698d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:21.385874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:21.386990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:21.387773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 94/190] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:21.388441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11b63c-4e1b-456e-9512-53cda69c698d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:21.396726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 97/191] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:21.397618 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb71cbc-051c-49a0-b4ad-4f6429affa3f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:21.440557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:21.493362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11b63c-4e1b-456e-9512-53cda69c698d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:21.498149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb71cbc-051c-49a0-b4ad-4f6429affa3f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:21.527337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b4e01-f8f5-49bb-90ac-657a4d3068c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:21.528134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b4e01-f8f5-49bb-90ac-657a4d3068c5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.528414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b4e01-f8f5-49bb-90ac-657a4d3068c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.528685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b4e01-f8f5-49bb-90ac-657a4d3068c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.529879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b4e01-f8f5-49bb-90ac-657a4d3068c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:21.531158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0aa99-c199-4cfb-b4da-48ed02920b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:21.531340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b4e01-f8f5-49bb-90ac-657a4d3068c5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:21.532015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0aa99-c199-4cfb-b4da-48ed02920b07 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.532291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0aa99-c199-4cfb-b4da-48ed02920b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.532558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0aa99-c199-4cfb-b4da-48ed02920b07 None admin] PATH_INFO: `/v3/users/33106ef743d34f3eb913a1369ad6ddb5` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.532964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 95/192] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:21.537582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b4e01-f8f5-49bb-90ac-657a4d3068c5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:21.549991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0aa99-c199-4cfb-b4da-48ed02920b07 None admin] RBAC: Authorizing `identity:get_user(user_id=33106ef743d34f3eb913a1369ad6ddb5)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:21.551459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0aa99-c199-4cfb-b4da-48ed02920b07 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:21.553182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 98/193] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:21 2022] GET /identity/v3/users/33106ef743d34f3eb913a1369ad6ddb5 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:21.553935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0aa99-c199-4cfb-b4da-48ed02920b07 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:21.593248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:21.638113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b4e01-f8f5-49bb-90ac-657a4d3068c5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:21.651842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:21.652202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0aa99-c199-4cfb-b4da-48ed02920b07 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:21.671936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ad961-9e0d-4dd0-942a-6903fdaffc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:21.672600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ad961-9e0d-4dd0-942a-6903fdaffc95 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.672926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ad961-9e0d-4dd0-942a-6903fdaffc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.673149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ad961-9e0d-4dd0-942a-6903fdaffc95 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.673924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ad961-9e0d-4dd0-942a-6903fdaffc95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:21.674225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ad961-9e0d-4dd0-942a-6903fdaffc95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:21.675599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ad961-9e0d-4dd0-942a-6903fdaffc95 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:21.684859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 96/194] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:21.684859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ad961-9e0d-4dd0-942a-6903fdaffc95 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:21.691297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b4dc9-e0b8-4d64-92d4-f07881b7c13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:21.692098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b4dc9-e0b8-4d64-92d4-f07881b7c13e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.692379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b4dc9-e0b8-4d64-92d4-f07881b7c13e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.692653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b4dc9-e0b8-4d64-92d4-f07881b7c13e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.703536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b4dc9-e0b8-4d64-92d4-f07881b7c13e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-eb3b4dc9-e0b8-4d64-92d4-f07881b7c13e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:21.712192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:21.713107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:21.713959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 99/195] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:21.714649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b4dc9-e0b8-4d64-92d4-f07881b7c13e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:21.787538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ad961-9e0d-4dd0-942a-6903fdaffc95 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:21.808396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:21.812501 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b4dc9-e0b8-4d64-92d4-f07881b7c13e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:21.823390 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55078a-edc0-43bd-a1ab-77564cfbf92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:21.824139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55078a-edc0-43bd-a1ab-77564cfbf92a None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.824427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55078a-edc0-43bd-a1ab-77564cfbf92a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.824702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55078a-edc0-43bd-a1ab-77564cfbf92a None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.825442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55078a-edc0-43bd-a1ab-77564cfbf92a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:21.826919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55078a-edc0-43bd-a1ab-77564cfbf92a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:21.847660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca827777-8753-4669-ab17-dbd524708c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:21.848307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca827777-8753-4669-ab17-dbd524708c71 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:21.848588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca827777-8753-4669-ab17-dbd524708c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:21.848861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca827777-8753-4669-ab17-dbd524708c71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:21.915351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca827777-8753-4669-ab17-dbd524708c71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:21.916851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca827777-8753-4669-ab17-dbd524708c71 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:21.990413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 97/196] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:21 2022] POST /identity/v3/users => generated 297 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:21.991525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55078a-edc0-43bd-a1ab-77564cfbf92a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:22.012114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:22.017648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 100/197] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:22.018344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca827777-8753-4669-ab17-dbd524708c71 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:22.027934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:22.106030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55078a-edc0-43bd-a1ab-77564cfbf92a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:22.130728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca827777-8753-4669-ab17-dbd524708c71 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:22.145658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea08cc-b002-4e29-97e3-5afa7de48148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:22.146549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea08cc-b002-4e29-97e3-5afa7de48148 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:22.146876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea08cc-b002-4e29-97e3-5afa7de48148 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:22.147873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea08cc-b002-4e29-97e3-5afa7de48148 None admin] PATH_INFO: `/v3/users/33106ef743d34f3eb913a1369ad6ddb5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:22.149750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea08cc-b002-4e29-97e3-5afa7de48148 None admin] RBAC: Authorizing `identity:get_user(user_id=33106ef743d34f3eb913a1369ad6ddb5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:22.150824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea08cc-b002-4e29-97e3-5afa7de48148 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:22.153372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 98/198] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:21 2022] GET /identity/v3/users/33106ef743d34f3eb913a1369ad6ddb5 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:22.153975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea08cc-b002-4e29-97e3-5afa7de48148 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:22.171961 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f55df9-0778-4697-8682-b1350f60c6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:22.172633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f55df9-0778-4697-8682-b1350f60c6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:22.172914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f55df9-0778-4697-8682-b1350f60c6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:22.173292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f55df9-0778-4697-8682-b1350f60c6c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:22.174093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f55df9-0778-4697-8682-b1350f60c6c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:22.174407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f55df9-0778-4697-8682-b1350f60c6c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:22.175801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f55df9-0778-4697-8682-b1350f60c6c5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:22.185377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 101/199] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:22.186071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f55df9-0778-4697-8682-b1350f60c6c5 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:22.253330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:22.277760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:22.281929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea08cc-b002-4e29-97e3-5afa7de48148 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:22.303145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f55df9-0778-4697-8682-b1350f60c6c5 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:22.321897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0599c3-a51b-42aa-a000-8bbabdfd8a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:22.323634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0599c3-a51b-42aa-a000-8bbabdfd8a3f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:22.323922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0599c3-a51b-42aa-a000-8bbabdfd8a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:22.324195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0599c3-a51b-42aa-a000-8bbabdfd8a3f None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:22.325452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0599c3-a51b-42aa-a000-8bbabdfd8a3f None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:22.326920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0599c3-a51b-42aa-a000-8bbabdfd8a3f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:22.333337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 99/200] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:22 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:22.335817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0599c3-a51b-42aa-a000-8bbabdfd8a3f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:22.346563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188419b-7656-45bb-a8a4-411167c680f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:22.346563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188419b-7656-45bb-a8a4-411167c680f8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:22.346563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188419b-7656-45bb-a8a4-411167c680f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:22.347059 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188419b-7656-45bb-a8a4-411167c680f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:22.427922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188419b-7656-45bb-a8a4-411167c680f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:22.429571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188419b-7656-45bb-a8a4-411167c680f8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:22.458968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0599c3-a51b-42aa-a000-8bbabdfd8a3f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:22.499797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:22.502716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabfb53-b622-4556-8550-10eb0d16801f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:22.503648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabfb53-b622-4556-8550-10eb0d16801f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:22.503941 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabfb53-b622-4556-8550-10eb0d16801f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:22.504216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabfb53-b622-4556-8550-10eb0d16801f None admin] PATH_INFO: `/v3/users/7fd5e6c20d9f4c02b862ed90c707d937` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:22.505511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabfb53-b622-4556-8550-10eb0d16801f None admin] RBAC: Authorizing `identity:get_user(user_id=7fd5e6c20d9f4c02b862ed90c707d937)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:22.506967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabfb53-b622-4556-8550-10eb0d16801f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:22.508700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 100/201] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:22 2022] GET /identity/v3/users/7fd5e6c20d9f4c02b862ed90c707d937 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:22.509514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabfb53-b622-4556-8550-10eb0d16801f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:22.546783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:22.551805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 102/202] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:22.552878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188419b-7656-45bb-a8a4-411167c680f8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:22.622135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabfb53-b622-4556-8550-10eb0d16801f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:22.658591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188419b-7656-45bb-a8a4-411167c680f8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:22.663663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf28e02-5016-44e9-9f8b-85fedea7eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:22.664333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf28e02-5016-44e9-9f8b-85fedea7eba9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:22.664621 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf28e02-5016-44e9-9f8b-85fedea7eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:22.664902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf28e02-5016-44e9-9f8b-85fedea7eba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:22.665790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf28e02-5016-44e9-9f8b-85fedea7eba9 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user.id=33106ef743d34f3eb913a1369ad6ddb5) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:22.666168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf28e02-5016-44e9-9f8b-85fedea7eba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:22.667619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf28e02-5016-44e9-9f8b-85fedea7eba9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:22.681349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 101/203] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:22 2022] GET /identity/v3/role_assignments?user.id=33106ef743d34f3eb913a1369ad6ddb5&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:22.682619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf28e02-5016-44e9-9f8b-85fedea7eba9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:22.695198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6f10b-0507-4c71-855c-faf434195fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:22.695870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6f10b-0507-4c71-855c-faf434195fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:22.696156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6f10b-0507-4c71-855c-faf434195fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:22.696436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6f10b-0507-4c71-855c-faf434195fb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:22.731161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:22.768873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6f10b-0507-4c71-855c-faf434195fb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:22.770408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6f10b-0507-4c71-855c-faf434195fb6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:22.874185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:22.879724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf28e02-5016-44e9-9f8b-85fedea7eba9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:22.887034 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 103/204] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:22.891697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6f10b-0507-4c71-855c-faf434195fb6 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:22.954883 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3617f-8fc7-47eb-ab73-5733400dba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:22.954883 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3617f-8fc7-47eb-ab73-5733400dba7e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:22.954883 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3617f-8fc7-47eb-ab73-5733400dba7e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:22.954883 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3617f-8fc7-47eb-ab73-5733400dba7e None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:22.956178 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3617f-8fc7-47eb-ab73-5733400dba7e None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:22.957719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3617f-8fc7-47eb-ab73-5733400dba7e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:22.960092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 102/205] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:22 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:22.973734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3617f-8fc7-47eb-ab73-5733400dba7e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:23.000468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:23.059205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6f10b-0507-4c71-855c-faf434195fb6 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:23.111496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141cd0e-57aa-4b58-9a06-410fc3a0c48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:23.112359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141cd0e-57aa-4b58-9a06-410fc3a0c48d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:23.112637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141cd0e-57aa-4b58-9a06-410fc3a0c48d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:23.112909 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141cd0e-57aa-4b58-9a06-410fc3a0c48d None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:23.115392 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141cd0e-57aa-4b58-9a06-410fc3a0c48d None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:23.116920 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141cd0e-57aa-4b58-9a06-410fc3a0c48d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:23.122500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 104/206] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:22 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:23.127851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141cd0e-57aa-4b58-9a06-410fc3a0c48d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:23.151585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3617f-8fc7-47eb-ab73-5733400dba7e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:23.193221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d428b3-5b44-4829-b113-da4e73cbc538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:23.193892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d428b3-5b44-4829-b113-da4e73cbc538 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:23.194168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d428b3-5b44-4829-b113-da4e73cbc538 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:23.194575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d428b3-5b44-4829-b113-da4e73cbc538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:23.195437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d428b3-5b44-4829-b113-da4e73cbc538 None admin] RBAC: Adding query filter params (role.id=41fd0da9b16d41c08992073473d87488, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user.id=33106ef743d34f3eb913a1369ad6ddb5) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:23.195742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d428b3-5b44-4829-b113-da4e73cbc538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:23.197343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d428b3-5b44-4829-b113-da4e73cbc538 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:23.204352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 103/207] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:22 2022] GET /identity/v3/role_assignments?user.id=33106ef743d34f3eb913a1369ad6ddb5&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=41fd0da9b16d41c08992073473d87488 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:23.261935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141cd0e-57aa-4b58-9a06-410fc3a0c48d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:23.291423 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:23.299848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26bdd1-d999-4d5b-96d6-a16801e44a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:23.300523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26bdd1-d999-4d5b-96d6-a16801e44a08 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:23.300802 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26bdd1-d999-4d5b-96d6-a16801e44a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:23.301073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26bdd1-d999-4d5b-96d6-a16801e44a08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:23.301947 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26bdd1-d999-4d5b-96d6-a16801e44a08 None admin] RBAC: Adding query filter params (role.id=4dc8f8c1b32c4c57805640c209616fde, scope.project.id=d76703b19c6c4521a9480e46863c1979, user.id=7fd5e6c20d9f4c02b862ed90c707d937) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:23.302256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26bdd1-d999-4d5b-96d6-a16801e44a08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:23.303706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26bdd1-d999-4d5b-96d6-a16801e44a08 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:23.310788 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 105/208] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:23 2022] GET /identity/v3/role_assignments?user.id=7fd5e6c20d9f4c02b862ed90c707d937&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=4dc8f8c1b32c4c57805640c209616fde => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:24.332782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27886349-87a5-405d-8c23-4895c2617963 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:24.333323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27886349-87a5-405d-8c23-4895c2617963 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:24.333323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27886349-87a5-405d-8c23-4895c2617963 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:24.335220 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 104/209] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:24.342547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86494c-adb1-4ce2-9e2a-8fced228d5e0 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:24.343430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86494c-adb1-4ce2-9e2a-8fced228d5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:24.343915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86494c-adb1-4ce2-9e2a-8fced228d5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:24.361296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-1a86494c-adb1-4ce2-9e2a-8fced228d5e0 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:24.362772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1a86494c-adb1-4ce2-9e2a-8fced228d5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:24.431406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 106/210] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:24.455110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27886349-87a5-405d-8c23-4895c2617963 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:24.491691 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:24.494591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602407c-9385-49fc-a4fc-f924727f8817 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:24.494897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602407c-9385-49fc-a4fc-f924727f8817 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:24.495141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602407c-9385-49fc-a4fc-f924727f8817 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:24.497125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 107/211] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:23:24.506630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c591fe-9868-4bac-b711-86e288c53d74 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:24.507293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c591fe-9868-4bac-b711-86e288c53d74 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:24.510479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c591fe-9868-4bac-b711-86e288c53d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:24.532655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-55c591fe-9868-4bac-b711-86e288c53d74 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:24.534726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-55c591fe-9868-4bac-b711-86e288c53d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:24.591470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27886349-87a5-405d-8c23-4895c2617963 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:24.627881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164254f0-bdae-41c6-86bd-300c23b493bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:24.628958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164254f0-bdae-41c6-86bd-300c23b493bf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:24.629276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 108/212] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:24.633534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164254f0-bdae-41c6-86bd-300c23b493bf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:24.634015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164254f0-bdae-41c6-86bd-300c23b493bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:24.645432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164254f0-bdae-41c6-86bd-300c23b493bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:24.653996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c591fe-9868-4bac-b711-86e288c53d74 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-164254f0-bdae-41c6-86bd-300c23b493bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:24.654133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:24.654997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:24.655815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 105/213] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:24.662328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164254f0-bdae-41c6-86bd-300c23b493bf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:24.729177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:24.765689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:24.776552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164254f0-bdae-41c6-86bd-300c23b493bf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:24.789422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c591fe-9868-4bac-b711-86e288c53d74 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:24.826739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6ad11-c2a3-4974-9c41-624017c20e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:24.827784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6ad11-c2a3-4974-9c41-624017c20e5c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:24.828540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6ad11-c2a3-4974-9c41-624017c20e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:24.828920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6ad11-c2a3-4974-9c41-624017c20e5c None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:24.829836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6ad11-c2a3-4974-9c41-624017c20e5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:24.830153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6ad11-c2a3-4974-9c41-624017c20e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:24.831520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dcf2b-9259-4ea2-8d40-061be0e9479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:24.831645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6ad11-c2a3-4974-9c41-624017c20e5c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:24.832632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dcf2b-9259-4ea2-8d40-061be0e9479a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:24.833115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dcf2b-9259-4ea2-8d40-061be0e9479a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:24.833611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dcf2b-9259-4ea2-8d40-061be0e9479a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:24.835051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dcf2b-9259-4ea2-8d40-061be0e9479a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:24.836752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dcf2b-9259-4ea2-8d40-061be0e9479a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:24.838872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 109/214] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:24.844521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 106/215] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:24.845743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dcf2b-9259-4ea2-8d40-061be0e9479a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:24.847013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6ad11-c2a3-4974-9c41-624017c20e5c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:24.972885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6ad11-c2a3-4974-9c41-624017c20e5c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:24.978752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dcf2b-9259-4ea2-8d40-061be0e9479a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:25.002103 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:25.011588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b6ed8-5dc0-497e-aa40-0b8d2adae16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:25.012794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b6ed8-5dc0-497e-aa40-0b8d2adae16f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.013137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b6ed8-5dc0-497e-aa40-0b8d2adae16f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.013453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b6ed8-5dc0-497e-aa40-0b8d2adae16f None admin] PATH_INFO: `/v3/users/729077e64d4a4de18a13d4ebb639e569` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.018897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb4f7f-9e44-4acc-a559-654fcee0bcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:25.019567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb4f7f-9e44-4acc-a559-654fcee0bcce None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.019850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb4f7f-9e44-4acc-a559-654fcee0bcce None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.020263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb4f7f-9e44-4acc-a559-654fcee0bcce None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.020993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb4f7f-9e44-4acc-a559-654fcee0bcce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:25.022481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb4f7f-9e44-4acc-a559-654fcee0bcce None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:25.028330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b6ed8-5dc0-497e-aa40-0b8d2adae16f None admin] RBAC: Authorizing `identity:get_user(user_id=729077e64d4a4de18a13d4ebb639e569)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:25.029825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b6ed8-5dc0-497e-aa40-0b8d2adae16f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:25.036000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 107/216] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:24 2022] GET /identity/v3/users/729077e64d4a4de18a13d4ebb639e569 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:25.037531 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d43e4c-bb17-4db7-90ba-6aac32bd5f27 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.038060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d43e4c-bb17-4db7-90ba-6aac32bd5f27 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.038573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d43e4c-bb17-4db7-90ba-6aac32bd5f27 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.040045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 108/217] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:25.040745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d43e4c-bb17-4db7-90ba-6aac32bd5f27 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:25.055615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:25.115964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 110/218] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:24 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:25.117652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50778f5-1e69-4e4e-90dd-1b1128632253 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.118192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50778f5-1e69-4e4e-90dd-1b1128632253 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.118721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50778f5-1e69-4e4e-90dd-1b1128632253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.148867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c50778f5-1e69-4e4e-90dd-1b1128632253 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:25.150557 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c50778f5-1e69-4e4e-90dd-1b1128632253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:25.150867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d43e4c-bb17-4db7-90ba-6aac32bd5f27 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:25.192313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903a784-3af6-4e3a-a987-54ff532bd993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:25.193510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903a784-3af6-4e3a-a987-54ff532bd993 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.194000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903a784-3af6-4e3a-a987-54ff532bd993 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.194488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903a784-3af6-4e3a-a987-54ff532bd993 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.241188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 111/219] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:25.262607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50778f5-1e69-4e4e-90dd-1b1128632253 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:25.286345 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903a784-3af6-4e3a-a987-54ff532bd993 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:25.288253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903a784-3af6-4e3a-a987-54ff532bd993 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:25.304427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:25.384495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50778f5-1e69-4e4e-90dd-1b1128632253 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:25.394344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:25.418152 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 109/220] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:25.424299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903a784-3af6-4e3a-a987-54ff532bd993 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:25.439789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dea5a-10d9-4685-ad8f-4f2ea7747a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:25.440928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dea5a-10d9-4685-ad8f-4f2ea7747a8c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.441265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dea5a-10d9-4685-ad8f-4f2ea7747a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.441540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dea5a-10d9-4685-ad8f-4f2ea7747a8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.458515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674dea5a-10d9-4685-ad8f-4f2ea7747a8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-674dea5a-10d9-4685-ad8f-4f2ea7747a8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:25.470161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:25.471298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:25.472297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 112/221] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:23:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:23:25.473611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b4377-4675-42f5-995f-cc852e6265fc None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.473901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b4377-4675-42f5-995f-cc852e6265fc None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.474207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b4377-4675-42f5-995f-cc852e6265fc None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.478091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 113/222] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:25.481671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b4377-4675-42f5-995f-cc852e6265fc None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:25.561651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903a784-3af6-4e3a-a987-54ff532bd993 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:25.589271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:25.607870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b4377-4675-42f5-995f-cc852e6265fc None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:25.621950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e400b-aafa-43b3-b65d-cb3412c309d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:25.623700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e400b-aafa-43b3-b65d-cb3412c309d9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.624164 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e400b-aafa-43b3-b65d-cb3412c309d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.624609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e400b-aafa-43b3-b65d-cb3412c309d9 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.626454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e400b-aafa-43b3-b65d-cb3412c309d9 None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:25.628404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e400b-aafa-43b3-b65d-cb3412c309d9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:25.632029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 110/223] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:25 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:25.633493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf896e-fdf5-4019-afda-aa56a6af6cc4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.633985 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf896e-fdf5-4019-afda-aa56a6af6cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.634439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf896e-fdf5-4019-afda-aa56a6af6cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.651023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56085a5d-8e0f-4f92-977f-1767fd56ad3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:25.651798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56085a5d-8e0f-4f92-977f-1767fd56ad3b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.652188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56085a5d-8e0f-4f92-977f-1767fd56ad3b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.652554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56085a5d-8e0f-4f92-977f-1767fd56ad3b None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.653440 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56085a5d-8e0f-4f92-977f-1767fd56ad3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:25.653863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56085a5d-8e0f-4f92-977f-1767fd56ad3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:25.655409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56085a5d-8e0f-4f92-977f-1767fd56ad3b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:25.656160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-15bf896e-fdf5-4019-afda-aa56a6af6cc4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:25.657743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-15bf896e-fdf5-4019-afda-aa56a6af6cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:25.665327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 114/224] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:25.670453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56085a5d-8e0f-4f92-977f-1767fd56ad3b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:25.703844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:25.757784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 111/225] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:25.759508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf896e-fdf5-4019-afda-aa56a6af6cc4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:25.795859 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56085a5d-8e0f-4f92-977f-1767fd56ad3b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:25.847199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da781eb8-9e93-461d-88d7-4b12a460a4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:25.848025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da781eb8-9e93-461d-88d7-4b12a460a4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.848314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da781eb8-9e93-461d-88d7-4b12a460a4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.848589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da781eb8-9e93-461d-88d7-4b12a460a4e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.849992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da781eb8-9e93-461d-88d7-4b12a460a4e9 None admin] RBAC: Adding query filter params (role.id=df5932e731294f7498a6983a730628bd, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user.id=729077e64d4a4de18a13d4ebb639e569) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:25.850315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da781eb8-9e93-461d-88d7-4b12a460a4e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:25.852590 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da781eb8-9e93-461d-88d7-4b12a460a4e9 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:25.869195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 115/226] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:25 2022] GET /identity/v3/role_assignments?user.id=729077e64d4a4de18a13d4ebb639e569&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=df5932e731294f7498a6983a730628bd => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:25.870078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da781eb8-9e93-461d-88d7-4b12a460a4e9 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:25.897609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf896e-fdf5-4019-afda-aa56a6af6cc4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:25.913450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:25.942714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56b8d3-f255-47f4-9c44-48610980762d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:25.943943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b8d3-f255-47f4-9c44-48610980762d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:25.944409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b8d3-f255-47f4-9c44-48610980762d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:25.945749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b8d3-f255-47f4-9c44-48610980762d None admin] PATH_INFO: `/v3/users/33106ef743d34f3eb913a1369ad6ddb5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:25.947394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56b8d3-f255-47f4-9c44-48610980762d None admin] RBAC: Authorizing `identity:get_user(user_id=33106ef743d34f3eb913a1369ad6ddb5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:25.949972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56b8d3-f255-47f4-9c44-48610980762d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:25.951957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 112/227] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:25 2022] GET /identity/v3/users/33106ef743d34f3eb913a1369ad6ddb5 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:25.958184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56b8d3-f255-47f4-9c44-48610980762d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:26.075414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56b8d3-f255-47f4-9c44-48610980762d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:26.119813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d5d7b-2b19-42a0-a3b2-2e9efad7758d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:26.121332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d5d7b-2b19-42a0-a3b2-2e9efad7758d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:26.121773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d5d7b-2b19-42a0-a3b2-2e9efad7758d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:26.122201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d5d7b-2b19-42a0-a3b2-2e9efad7758d None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:26.124143 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d5d7b-2b19-42a0-a3b2-2e9efad7758d None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:26.126017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d5d7b-2b19-42a0-a3b2-2e9efad7758d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:26.128121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 113/228] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:25 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:26.129042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:26.134469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d5d7b-2b19-42a0-a3b2-2e9efad7758d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:26.140098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da781eb8-9e93-461d-88d7-4b12a460a4e9 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:26.193693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:26.245581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d5d7b-2b19-42a0-a3b2-2e9efad7758d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:26.265912 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ec9c9-1f31-47c3-89b6-46ffab74c543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:26.266737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ec9c9-1f31-47c3-89b6-46ffab74c543 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:26.267024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ec9c9-1f31-47c3-89b6-46ffab74c543 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:26.267304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ec9c9-1f31-47c3-89b6-46ffab74c543 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:26.283316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce337-ea08-4b83-8d99-7fd7b46216bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:26.284426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce337-ea08-4b83-8d99-7fd7b46216bc None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:26.284902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce337-ea08-4b83-8d99-7fd7b46216bc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:26.286698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce337-ea08-4b83-8d99-7fd7b46216bc None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/users/33106ef743d34f3eb913a1369ad6ddb5/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:26.290381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ce337-ea08-4b83-8d99-7fd7b46216bc None admin] RBAC: Authorizing `identity:create_grant(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user_id=33106ef743d34f3eb913a1369ad6ddb5, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:26.294571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ce337-ea08-4b83-8d99-7fd7b46216bc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:26.298451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ec9c9-1f31-47c3-89b6-46ffab74c543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-4e3ec9c9-1f31-47c3-89b6-46ffab74c543 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:26.306821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:26.307873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:26.308851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:26.312932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 116/229] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:26.319277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ec9c9-1f31-47c3-89b6-46ffab74c543 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:26.427684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ec9c9-1f31-47c3-89b6-46ffab74c543 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:26.436025 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 114/230] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:23:26 2022] PUT /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/users/33106ef743d34f3eb913a1369ad6ddb5/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:23:26.473066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd23ef0-5bc0-4b4c-9a46-5825d808da0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:26.473994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd23ef0-5bc0-4b4c-9a46-5825d808da0e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:26.474418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd23ef0-5bc0-4b4c-9a46-5825d808da0e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:26.474829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd23ef0-5bc0-4b4c-9a46-5825d808da0e None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:26.475780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd23ef0-5bc0-4b4c-9a46-5825d808da0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:26.476254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd23ef0-5bc0-4b4c-9a46-5825d808da0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:26.477831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd23ef0-5bc0-4b4c-9a46-5825d808da0e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:26.487484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 117/231] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:26.493193 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce337-ea08-4b83-8d99-7fd7b46216bc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:26.640107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce337-ea08-4b83-8d99-7fd7b46216bc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:26.651412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:26.685827 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b707c61-935c-4de1-bd64-4de431a6e37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:26.686772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b707c61-935c-4de1-bd64-4de431a6e37c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:26.687054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b707c61-935c-4de1-bd64-4de431a6e37c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:26.687328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b707c61-935c-4de1-bd64-4de431a6e37c None admin] PATH_INFO: `/v3/users/33106ef743d34f3eb913a1369ad6ddb5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:26.689477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b707c61-935c-4de1-bd64-4de431a6e37c None admin] RBAC: Authorizing `identity:get_user(user_id=33106ef743d34f3eb913a1369ad6ddb5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:26.701596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b707c61-935c-4de1-bd64-4de431a6e37c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:26.701596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 115/232] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:26 2022] GET /identity/v3/users/33106ef743d34f3eb913a1369ad6ddb5 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:26.701778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd23ef0-5bc0-4b4c-9a46-5825d808da0e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:26.720055 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:26.814401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd23ef0-5bc0-4b4c-9a46-5825d808da0e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:26.860559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c521f-165e-4950-ba31-51b4e3b56783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:26.861866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c521f-165e-4950-ba31-51b4e3b56783 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:26.862146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c521f-165e-4950-ba31-51b4e3b56783 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:26.862418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c521f-165e-4950-ba31-51b4e3b56783 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:26.863633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c521f-165e-4950-ba31-51b4e3b56783 None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:26.865088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c521f-165e-4950-ba31-51b4e3b56783 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:26.867410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 118/233] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:26 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:26.873714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b707c61-935c-4de1-bd64-4de431a6e37c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:27.025515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b707c61-935c-4de1-bd64-4de431a6e37c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:27.068940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d88984-3d10-4496-b53e-7c622604948e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:27.070613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d88984-3d10-4496-b53e-7c622604948e None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.070896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d88984-3d10-4496-b53e-7c622604948e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.071202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d88984-3d10-4496-b53e-7c622604948e None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/users/33106ef743d34f3eb913a1369ad6ddb5/roles/41fd0da9b16d41c08992073473d87488` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.073546 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d88984-3d10-4496-b53e-7c622604948e None admin] RBAC: Authorizing `identity:create_grant(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user_id=33106ef743d34f3eb913a1369ad6ddb5, role_id=41fd0da9b16d41c08992073473d87488)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:27.075069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d88984-3d10-4496-b53e-7c622604948e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:27.199115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 116/234] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:23:26 2022] PUT /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/users/33106ef743d34f3eb913a1369ad6ddb5/roles/41fd0da9b16d41c08992073473d87488 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:27.642726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7311d2-3e6c-492b-8229-67daf738a03e None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.642726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7311d2-3e6c-492b-8229-67daf738a03e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.642726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7311d2-3e6c-492b-8229-67daf738a03e None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.643698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 119/235] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:27.647958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93adca-046e-4081-ad72-d1e4787bb534 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.648709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93adca-046e-4081-ad72-d1e4787bb534 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.649122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93adca-046e-4081-ad72-d1e4787bb534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.670133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3b93adca-046e-4081-ad72-d1e4787bb534 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:27.671546 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3b93adca-046e-4081-ad72-d1e4787bb534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:27.703987 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:27.749856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 117/236] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:27.781443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7311d2-3e6c-492b-8229-67daf738a03e None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:27.800095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115144f-da6b-40a9-a314-c5d859681299 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.800801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115144f-da6b-40a9-a314-c5d859681299 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.800801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115144f-da6b-40a9-a314-c5d859681299 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.802042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 118/237] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:27.808495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bb519-292e-4dda-ad87-a6ac33750cb3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.808778 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bb519-292e-4dda-ad87-a6ac33750cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.809053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bb519-292e-4dda-ad87-a6ac33750cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.828650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d01bb519-292e-4dda-ad87-a6ac33750cb3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:27.830191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d01bb519-292e-4dda-ad87-a6ac33750cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:27.942715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:27.962959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7311d2-3e6c-492b-8229-67daf738a03e None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:27.967684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 119/238] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:27.999702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bb519-292e-4dda-ad87-a6ac33750cb3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:28.029022 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a389365-9a67-4114-9a04-d7808ccd8b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:28.030190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a389365-9a67-4114-9a04-d7808ccd8b27 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:28.030694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a389365-9a67-4114-9a04-d7808ccd8b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:28.031206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a389365-9a67-4114-9a04-d7808ccd8b27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:28.043071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a389365-9a67-4114-9a04-d7808ccd8b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-0a389365-9a67-4114-9a04-d7808ccd8b27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:28.051354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:28.052298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:28.053093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:28.056498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 120/239] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:28.062882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a389365-9a67-4114-9a04-d7808ccd8b27 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:28.189292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:28.206825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bb519-292e-4dda-ad87-a6ac33750cb3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:28.259747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a389365-9a67-4114-9a04-d7808ccd8b27 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:28.274223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586580-a2ff-4083-8a86-c6871f54097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:28.275370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586580-a2ff-4083-8a86-c6871f54097b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:28.275897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586580-a2ff-4083-8a86-c6871f54097b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:28.276370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586580-a2ff-4083-8a86-c6871f54097b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:28.279658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586580-a2ff-4083-8a86-c6871f54097b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:28.281636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586580-a2ff-4083-8a86-c6871f54097b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:28.288402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 120/240] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:28.296454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586580-a2ff-4083-8a86-c6871f54097b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:28.328117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce5824-4856-428e-a57d-97aa2d2505f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:28.329142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce5824-4856-428e-a57d-97aa2d2505f7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:28.329675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce5824-4856-428e-a57d-97aa2d2505f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:28.330301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce5824-4856-428e-a57d-97aa2d2505f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:28.331266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce5824-4856-428e-a57d-97aa2d2505f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:28.331959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce5824-4856-428e-a57d-97aa2d2505f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:28.333719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce5824-4856-428e-a57d-97aa2d2505f7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:28.345228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 121/241] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:28.354360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce5824-4856-428e-a57d-97aa2d2505f7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:28.369534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:28.446341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586580-a2ff-4083-8a86-c6871f54097b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:28.522369 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8676da-4d5c-4a42-ad86-133eaeefe68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:28.523549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8676da-4d5c-4a42-ad86-133eaeefe68b None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:28.524174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8676da-4d5c-4a42-ad86-133eaeefe68b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:28.525482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8676da-4d5c-4a42-ad86-133eaeefe68b None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:28.526724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8676da-4d5c-4a42-ad86-133eaeefe68b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:28.529260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8676da-4d5c-4a42-ad86-133eaeefe68b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:28.541874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce5824-4856-428e-a57d-97aa2d2505f7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:28.592442 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643f5d6-ca99-4d66-b223-32d10613f76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:28.593800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643f5d6-ca99-4d66-b223-32d10613f76b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:28.594317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643f5d6-ca99-4d66-b223-32d10613f76b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:28.594831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643f5d6-ca99-4d66-b223-32d10613f76b None admin] PATH_INFO: `/v3/users/e63b482b343e45eb828a1e4eb598401a` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:28.611422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643f5d6-ca99-4d66-b223-32d10613f76b None admin] RBAC: Authorizing `identity:get_user(user_id=e63b482b343e45eb828a1e4eb598401a)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:28.614837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643f5d6-ca99-4d66-b223-32d10613f76b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:28.615504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 122/242] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:28 2022] GET /identity/v3/users/e63b482b343e45eb828a1e4eb598401a => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:28.621749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643f5d6-ca99-4d66-b223-32d10613f76b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:28.638521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 121/243] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:28 2022] POST /identity/v3/users => generated 293 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:28.675067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cde69-5869-4754-abd6-6b02022da509 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:28.675676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cde69-5869-4754-abd6-6b02022da509 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:28.675676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cde69-5869-4754-abd6-6b02022da509 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:28.677368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 122/244] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:28.684963 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2b89b-c80d-4a30-8b5c-abfd3e69eca8 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:28.685360 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2b89b-c80d-4a30-8b5c-abfd3e69eca8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:28.685717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2b89b-c80d-4a30-8b5c-abfd3e69eca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:28.719686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:28.727039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7ec2b89b-c80d-4a30-8b5c-abfd3e69eca8 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:28.728240 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7ec2b89b-c80d-4a30-8b5c-abfd3e69eca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:28.750605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:28.779859 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643f5d6-ca99-4d66-b223-32d10613f76b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:28.827406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b78603-ed23-40a2-bb60-029c1912356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:28.828671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b78603-ed23-40a2-bb60-029c1912356f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:28.829138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b78603-ed23-40a2-bb60-029c1912356f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:28.829631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b78603-ed23-40a2-bb60-029c1912356f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:28.871715 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 123/245] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:28.895268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2b89b-c80d-4a30-8b5c-abfd3e69eca8 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:28.924826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b78603-ed23-40a2-bb60-029c1912356f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:28.926365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b78603-ed23-40a2-bb60-029c1912356f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:28.991753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:29.017126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2b89b-c80d-4a30-8b5c-abfd3e69eca8 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:29.033466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 123/246] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:29.039499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b78603-ed23-40a2-bb60-029c1912356f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:29.060084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1ae14-ed7f-4280-bb5f-6920230fd7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:29.068564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1ae14-ed7f-4280-bb5f-6920230fd7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.069140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1ae14-ed7f-4280-bb5f-6920230fd7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.069631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1ae14-ed7f-4280-bb5f-6920230fd7b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.087388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1ae14-ed7f-4280-bb5f-6920230fd7b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-16c1ae14-ed7f-4280-bb5f-6920230fd7b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:29.095885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:29.096901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:29.097744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:29.098350 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 124/247] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:29.100714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712c55c-26d3-4f1b-b3b9-d8386bda3d6e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.100714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712c55c-26d3-4f1b-b3b9-d8386bda3d6e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.100714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712c55c-26d3-4f1b-b3b9-d8386bda3d6e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.102533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 125/248] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:29.108166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712c55c-26d3-4f1b-b3b9-d8386bda3d6e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:29.121074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:29.173657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b78603-ed23-40a2-bb60-029c1912356f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:29.215743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1c011-c522-48d1-ab0b-6612cf0c4891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:29.217338 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c011-c522-48d1-ab0b-6612cf0c4891 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.217842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c011-c522-48d1-ab0b-6612cf0c4891 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.218313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c011-c522-48d1-ab0b-6612cf0c4891 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.220075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1c011-c522-48d1-ab0b-6612cf0c4891 None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:29.222232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1c011-c522-48d1-ab0b-6612cf0c4891 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:29.225082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 124/249] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:29 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:29.226593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b7ca6-a13c-4f38-a706-c565def8d149 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.227076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b7ca6-a13c-4f38-a706-c565def8d149 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.227555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b7ca6-a13c-4f38-a706-c565def8d149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.248818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-5c5b7ca6-a13c-4f38-a706-c565def8d149 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:29.250215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5c5b7ca6-a13c-4f38-a706-c565def8d149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:29.252013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712c55c-26d3-4f1b-b3b9-d8386bda3d6e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:29.286813 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:29.303333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4950f-2b98-4fd9-b53a-bb86ca8bafe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:29.304049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4950f-2b98-4fd9-b53a-bb86ca8bafe8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.304335 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4950f-2b98-4fd9-b53a-bb86ca8bafe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.304614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4950f-2b98-4fd9-b53a-bb86ca8bafe8 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.305404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4950f-2b98-4fd9-b53a-bb86ca8bafe8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:29.305713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4950f-2b98-4fd9-b53a-bb86ca8bafe8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:29.307484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4950f-2b98-4fd9-b53a-bb86ca8bafe8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:29.317956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 126/250] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:29.318528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4950f-2b98-4fd9-b53a-bb86ca8bafe8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:29.357366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 125/251] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:29.358555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b7ca6-a13c-4f38-a706-c565def8d149 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:29.415131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:29.456833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4950f-2b98-4fd9-b53a-bb86ca8bafe8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:29.499912 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b7ca6-a13c-4f38-a706-c565def8d149 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:29.503791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8964c23-dea7-41b8-ad26-0e872415561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:29.504706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8964c23-dea7-41b8-ad26-0e872415561b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.505232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8964c23-dea7-41b8-ad26-0e872415561b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.505706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8964c23-dea7-41b8-ad26-0e872415561b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.506772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8964c23-dea7-41b8-ad26-0e872415561b None admin] RBAC: Adding query filter params (role.id=4dc8f8c1b32c4c57805640c209616fde, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user.id=e63b482b343e45eb828a1e4eb598401a) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:29.507825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8964c23-dea7-41b8-ad26-0e872415561b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:29.510006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8964c23-dea7-41b8-ad26-0e872415561b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:29.518425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 127/252] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:29 2022] GET /identity/v3/role_assignments?user.id=e63b482b343e45eb828a1e4eb598401a&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=4dc8f8c1b32c4c57805640c209616fde => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:29.519554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8964c23-dea7-41b8-ad26-0e872415561b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:29.573842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f7790-9808-446d-8acb-5891f376c7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:29.574699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f7790-9808-446d-8acb-5891f376c7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.574982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f7790-9808-446d-8acb-5891f376c7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.575308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f7790-9808-446d-8acb-5891f376c7e9 None admin] PATH_INFO: `/v3/users/729077e64d4a4de18a13d4ebb639e569` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.580300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f7790-9808-446d-8acb-5891f376c7e9 None admin] RBAC: Authorizing `identity:get_user(user_id=729077e64d4a4de18a13d4ebb639e569)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:29.581905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f7790-9808-446d-8acb-5891f376c7e9 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:29.583656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 126/253] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:29 2022] GET /identity/v3/users/729077e64d4a4de18a13d4ebb639e569 => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:29.585628 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c2479-ce5d-4e67-b7dc-41270cf283ef None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.585921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c2479-ce5d-4e67-b7dc-41270cf283ef None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.586199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c2479-ce5d-4e67-b7dc-41270cf283ef None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.587470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 127/254] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:29.594255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c2479-ce5d-4e67-b7dc-41270cf283ef None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:29.646742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8964c23-dea7-41b8-ad26-0e872415561b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:29.653558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:29.709650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346dcdd-18af-4db7-b871-08607cd08e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:29.711735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346dcdd-18af-4db7-b871-08607cd08e8a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.712410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346dcdd-18af-4db7-b871-08607cd08e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.714310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346dcdd-18af-4db7-b871-08607cd08e8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.718831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:29.736892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346dcdd-18af-4db7-b871-08607cd08e8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-d346dcdd-18af-4db7-b871-08607cd08e8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:29.748736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:29.749786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:29.750620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 128/255] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:23:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:29.751275 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14797b-cc36-4467-8022-608b43e079da None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.752158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14797b-cc36-4467-8022-608b43e079da None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.752158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14797b-cc36-4467-8022-608b43e079da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.770323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ad14797b-cc36-4467-8022-608b43e079da None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:29.771509 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ad14797b-cc36-4467-8022-608b43e079da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:29.815633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c2479-ce5d-4e67-b7dc-41270cf283ef None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:29.883074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 129/256] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:29.883844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14797b-cc36-4467-8022-608b43e079da None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:29.897503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6b7d6-73c8-41c2-a76a-a99a283deb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:29.898895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6b7d6-73c8-41c2-a76a-a99a283deb7b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:29.899183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6b7d6-73c8-41c2-a76a-a99a283deb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:29.899458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6b7d6-73c8-41c2-a76a-a99a283deb7b None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:29.906042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6b7d6-73c8-41c2-a76a-a99a283deb7b None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:29.907555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6b7d6-73c8-41c2-a76a-a99a283deb7b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:29.909704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 128/257] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:29 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:29.917478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6b7d6-73c8-41c2-a76a-a99a283deb7b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:30.050047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:30.056330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6b7d6-73c8-41c2-a76a-a99a283deb7b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:30.069130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14797b-cc36-4467-8022-608b43e079da None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:30.092685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d3893-6e64-4878-95eb-2dea819d9bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:30.093609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d3893-6e64-4878-95eb-2dea819d9bc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:30.093891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d3893-6e64-4878-95eb-2dea819d9bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:30.094174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d3893-6e64-4878-95eb-2dea819d9bc9 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/users/729077e64d4a4de18a13d4ebb639e569/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:30.096213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d3893-6e64-4878-95eb-2dea819d9bc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user_id=729077e64d4a4de18a13d4ebb639e569, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:30.097752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d3893-6e64-4878-95eb-2dea819d9bc9 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:30.117593 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:30.128549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fb4da-6b09-4370-b610-862bd7c789df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:30.129256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fb4da-6b09-4370-b610-862bd7c789df None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:30.129592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fb4da-6b09-4370-b610-862bd7c789df None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:30.129872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fb4da-6b09-4370-b610-862bd7c789df None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:30.130639 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fb4da-6b09-4370-b610-862bd7c789df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:30.130946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fb4da-6b09-4370-b610-862bd7c789df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:30.132452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fb4da-6b09-4370-b610-862bd7c789df None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:30.146162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 130/258] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:30.146162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fb4da-6b09-4370-b610-862bd7c789df None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:30.170905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 129/259] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:23:29 2022] PUT /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/users/729077e64d4a4de18a13d4ebb639e569/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:30.171770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d3893-6e64-4878-95eb-2dea819d9bc9 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:30.256133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fb4da-6b09-4370-b610-862bd7c789df None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:30.293770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09f2c4-3ad1-4d1a-9986-326d6e054e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:30.294734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09f2c4-3ad1-4d1a-9986-326d6e054e04 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:30.295017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09f2c4-3ad1-4d1a-9986-326d6e054e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:30.295287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09f2c4-3ad1-4d1a-9986-326d6e054e04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:30.318726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09f2c4-3ad1-4d1a-9986-326d6e054e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-db09f2c4-3ad1-4d1a-9986-326d6e054e04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:30.336764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:30.337803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:30.338603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:30.339231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 131/260] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:30.346836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09f2c4-3ad1-4d1a-9986-326d6e054e04 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:30.361686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d3893-6e64-4878-95eb-2dea819d9bc9 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:30.407134 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba4dd0-a432-4e54-8df0-73a7429687b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:30.407937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba4dd0-a432-4e54-8df0-73a7429687b5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:30.408216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba4dd0-a432-4e54-8df0-73a7429687b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:30.408491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba4dd0-a432-4e54-8df0-73a7429687b5 None admin] PATH_INFO: `/v3/users/33106ef743d34f3eb913a1369ad6ddb5` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:30.410060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba4dd0-a432-4e54-8df0-73a7429687b5 None admin] RBAC: Authorizing `identity:get_user(user_id=33106ef743d34f3eb913a1369ad6ddb5)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:30.411829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba4dd0-a432-4e54-8df0-73a7429687b5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:30.413567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 130/261] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:30 2022] GET /identity/v3/users/33106ef743d34f3eb913a1369ad6ddb5 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:30.422769 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba4dd0-a432-4e54-8df0-73a7429687b5 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:30.454255 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:30.475101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:30.580679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba4dd0-a432-4e54-8df0-73a7429687b5 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:30.583498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09f2c4-3ad1-4d1a-9986-326d6e054e04 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:30.627750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37326ad9-0c11-4d4c-b0bb-39a03d042178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:30.628529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37326ad9-0c11-4d4c-b0bb-39a03d042178 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:30.628816 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37326ad9-0c11-4d4c-b0bb-39a03d042178 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:30.629092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37326ad9-0c11-4d4c-b0bb-39a03d042178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:30.694079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f692a7-46f2-48f7-be38-36cbdba45fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:30.694711 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f692a7-46f2-48f7-be38-36cbdba45fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:30.694991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f692a7-46f2-48f7-be38-36cbdba45fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:30.695267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f692a7-46f2-48f7-be38-36cbdba45fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:30.696030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f692a7-46f2-48f7-be38-36cbdba45fc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:30.696334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f692a7-46f2-48f7-be38-36cbdba45fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:30.697751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f692a7-46f2-48f7-be38-36cbdba45fc1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:30.710551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37326ad9-0c11-4d4c-b0bb-39a03d042178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:30.712083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37326ad9-0c11-4d4c-b0bb-39a03d042178 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:30.716529 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 132/262] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:30.722078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f692a7-46f2-48f7-be38-36cbdba45fc1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:30.739537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:30.822804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 131/263] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:30.830122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37326ad9-0c11-4d4c-b0bb-39a03d042178 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:30.913901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f692a7-46f2-48f7-be38-36cbdba45fc1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:30.933691 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:30.984082 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551948c-153b-4633-9c03-c6cc0987c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:30.985537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551948c-153b-4633-9c03-c6cc0987c293 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:30.986010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551948c-153b-4633-9c03-c6cc0987c293 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:30.986424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551948c-153b-4633-9c03-c6cc0987c293 None admin] PATH_INFO: `/v3/users/33106ef743d34f3eb913a1369ad6ddb5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:30.988046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551948c-153b-4633-9c03-c6cc0987c293 None admin] RBAC: Authorizing `identity:get_user(user_id=33106ef743d34f3eb913a1369ad6ddb5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:30.994465 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551948c-153b-4633-9c03-c6cc0987c293 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:30.996582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 133/264] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:30 2022] GET /identity/v3/users/33106ef743d34f3eb913a1369ad6ddb5 => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:31.002055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551948c-153b-4633-9c03-c6cc0987c293 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:31.082929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37326ad9-0c11-4d4c-b0bb-39a03d042178 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:31.130288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:31.156788 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda9faa-b0ff-4126-a0f8-f829474f0fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:31.157688 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9faa-b0ff-4126-a0f8-f829474f0fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:31.157970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9faa-b0ff-4126-a0f8-f829474f0fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:31.158248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9faa-b0ff-4126-a0f8-f829474f0fc3 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:31.161084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda9faa-b0ff-4126-a0f8-f829474f0fc3 None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:31.162611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda9faa-b0ff-4126-a0f8-f829474f0fc3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:31.167885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 132/265] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:30 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:31.173467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda9faa-b0ff-4126-a0f8-f829474f0fc3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:31.214644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551948c-153b-4633-9c03-c6cc0987c293 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:31.297093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a60ca-87fc-497e-a1ee-a120597667b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:31.298268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a60ca-87fc-497e-a1ee-a120597667b2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:31.298750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a60ca-87fc-497e-a1ee-a120597667b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:31.299210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a60ca-87fc-497e-a1ee-a120597667b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:31.401640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:31.417868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda9faa-b0ff-4126-a0f8-f829474f0fc3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:31.444810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a60ca-87fc-497e-a1ee-a120597667b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:31.446359 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a60ca-87fc-497e-a1ee-a120597667b2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:31.495648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969e856-1fe0-4cbe-a357-3d56c54ea741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:31.496308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969e856-1fe0-4cbe-a357-3d56c54ea741 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:31.496691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969e856-1fe0-4cbe-a357-3d56c54ea741 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:31.496975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969e856-1fe0-4cbe-a357-3d56c54ea741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:31.497854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969e856-1fe0-4cbe-a357-3d56c54ea741 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user.id=33106ef743d34f3eb913a1369ad6ddb5) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:31.498172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969e856-1fe0-4cbe-a357-3d56c54ea741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:31.499639 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969e856-1fe0-4cbe-a357-3d56c54ea741 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:31.514091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 133/266] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:31 2022] GET /identity/v3/role_assignments?user.id=33106ef743d34f3eb913a1369ad6ddb5&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:31.594324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 134/267] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 11:23:31.602573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969e856-1fe0-4cbe-a357-3d56c54ea741 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:31.684228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:31.741279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969e856-1fe0-4cbe-a357-3d56c54ea741 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:31.780781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a5712-e4d0-4d7d-89fc-7b546674db0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:31.781676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a5712-e4d0-4d7d-89fc-7b546674db0e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:31.781955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a5712-e4d0-4d7d-89fc-7b546674db0e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:31.782223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a5712-e4d0-4d7d-89fc-7b546674db0e None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:31.783411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a5712-e4d0-4d7d-89fc-7b546674db0e None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:31.784860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a5712-e4d0-4d7d-89fc-7b546674db0e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:31.786774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 134/268] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:31 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:31.793762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a60ca-87fc-497e-a1ee-a120597667b2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:31.885407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:31.908910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a60ca-87fc-497e-a1ee-a120597667b2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:31.953304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847f75c-feee-4ff1-a0e7-e231ef46fe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:31.953969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847f75c-feee-4ff1-a0e7-e231ef46fe33 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:31.954251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847f75c-feee-4ff1-a0e7-e231ef46fe33 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:31.954526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847f75c-feee-4ff1-a0e7-e231ef46fe33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:31.955364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847f75c-feee-4ff1-a0e7-e231ef46fe33 None admin] RBAC: Adding query filter params (role.id=41fd0da9b16d41c08992073473d87488, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user.id=33106ef743d34f3eb913a1369ad6ddb5) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:31.955674 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847f75c-feee-4ff1-a0e7-e231ef46fe33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:31.957114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847f75c-feee-4ff1-a0e7-e231ef46fe33 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:31.964086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 135/269] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:31 2022] GET /identity/v3/role_assignments?user.id=33106ef743d34f3eb913a1369ad6ddb5&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=41fd0da9b16d41c08992073473d87488 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:31.971561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5522f-dd49-4816-96f6-8e62533017d5 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:31.972491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5522f-dd49-4816-96f6-8e62533017d5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:31.972962 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5522f-dd49-4816-96f6-8e62533017d5 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:31.974548 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 135/270] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:31.981528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ddc3a-25e3-4b12-aa83-b72686d2d7ce None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:31.981824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ddc3a-25e3-4b12-aa83-b72686d2d7ce None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:31.982090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ddc3a-25e3-4b12-aa83-b72686d2d7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.004107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a30ddc3a-25e3-4b12-aa83-b72686d2d7ce None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:32.005344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a30ddc3a-25e3-4b12-aa83-b72686d2d7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:32.060797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcb8c0-ee58-4821-a3c8-8ec6a28a1828 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.061507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcb8c0-ee58-4821-a3c8-8ec6a28a1828 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.061994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcb8c0-ee58-4821-a3c8-8ec6a28a1828 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.064000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 136/271] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:32.070505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412f711-8eaa-4a39-a957-87717394d52d None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.070840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412f711-8eaa-4a39-a957-87717394d52d None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.071070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412f711-8eaa-4a39-a957-87717394d52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.094028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 136/272] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:32.113418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-d412f711-8eaa-4a39-a957-87717394d52d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:32.114647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d412f711-8eaa-4a39-a957-87717394d52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:32.151245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ddc3a-25e3-4b12-aa83-b72686d2d7ce None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.238594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:32.241898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 137/273] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:32.243726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8eca5-a902-4b9c-a339-78ad8cfde42b None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.244157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8eca5-a902-4b9c-a339-78ad8cfde42b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.244561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8eca5-a902-4b9c-a339-78ad8cfde42b None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.247168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 138/274] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:32.253331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c5055-71e9-4f04-86da-76555834e648 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.253912 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c5055-71e9-4f04-86da-76555834e648 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.254368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c5055-71e9-4f04-86da-76555834e648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.275554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ddc3a-25e3-4b12-aa83-b72686d2d7ce None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.276543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c51c5055-71e9-4f04-86da-76555834e648 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:32.278149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c51c5055-71e9-4f04-86da-76555834e648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:32.318502 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6593d-c428-4fa2-9fc3-e10d07d7e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.319584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6593d-c428-4fa2-9fc3-e10d07d7e25c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.320055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6593d-c428-4fa2-9fc3-e10d07d7e25c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.320524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6593d-c428-4fa2-9fc3-e10d07d7e25c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.332817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6593d-c428-4fa2-9fc3-e10d07d7e25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:32.339065 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b8f6593d-c428-4fa2-9fc3-e10d07d7e25c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:32.342398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:32.343306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:32.344100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:32.345711 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 137/275] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:23:32.345711 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6593d-c428-4fa2-9fc3-e10d07d7e25c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.364376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 139/276] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:32.365274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c5055-71e9-4f04-86da-76555834e648 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.468599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6593d-c428-4fa2-9fc3-e10d07d7e25c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.486849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c5055-71e9-4f04-86da-76555834e648 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.500764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:32.506948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764dcde-8b57-421a-bf14-08ba633cd652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.507739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764dcde-8b57-421a-bf14-08ba633cd652 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.508018 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764dcde-8b57-421a-bf14-08ba633cd652 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.508417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764dcde-8b57-421a-bf14-08ba633cd652 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.514919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764dcde-8b57-421a-bf14-08ba633cd652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:32.514919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764dcde-8b57-421a-bf14-08ba633cd652 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:32.514919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 138/277] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:32.514919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764dcde-8b57-421a-bf14-08ba633cd652 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.534217 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae6288-3dee-4cba-9aa1-95d5fb53cb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.534893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6288-3dee-4cba-9aa1-95d5fb53cb19 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.535179 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6288-3dee-4cba-9aa1-95d5fb53cb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.535457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6288-3dee-4cba-9aa1-95d5fb53cb19 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.536219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae6288-3dee-4cba-9aa1-95d5fb53cb19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:32.536540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae6288-3dee-4cba-9aa1-95d5fb53cb19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:32.538047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae6288-3dee-4cba-9aa1-95d5fb53cb19 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:32.546854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 140/278] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:32.547470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae6288-3dee-4cba-9aa1-95d5fb53cb19 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.595877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:32.634780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764dcde-8b57-421a-bf14-08ba633cd652 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.653573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae6288-3dee-4cba-9aa1-95d5fb53cb19 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.670670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff140700-22f6-4465-8ce4-bf3508ced8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.671466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff140700-22f6-4465-8ce4-bf3508ced8fe None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.671756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff140700-22f6-4465-8ce4-bf3508ced8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.672028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff140700-22f6-4465-8ce4-bf3508ced8fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.682531 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff140700-22f6-4465-8ce4-bf3508ced8fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-ff140700-22f6-4465-8ce4-bf3508ced8fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:32.690202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:32.691109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:32.691916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:32.692513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba366883-d3ab-493c-824f-18290b12615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.692513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba366883-d3ab-493c-824f-18290b12615b None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.692513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba366883-d3ab-493c-824f-18290b12615b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.692513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba366883-d3ab-493c-824f-18290b12615b None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.692768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba366883-d3ab-493c-824f-18290b12615b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:32.693872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba366883-d3ab-493c-824f-18290b12615b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:32.696379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 139/279] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:32.697256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff140700-22f6-4465-8ce4-bf3508ced8fe None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.756267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:32.798409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff140700-22f6-4465-8ce4-bf3508ced8fe None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.841192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 141/280] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:32 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:32.841192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba366883-d3ab-493c-824f-18290b12615b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.845372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c2238-33a6-4d54-ae7a-0a2d6bd675d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.846290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c2238-33a6-4d54-ae7a-0a2d6bd675d6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.846580 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c2238-33a6-4d54-ae7a-0a2d6bd675d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.846859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c2238-33a6-4d54-ae7a-0a2d6bd675d6 None admin] PATH_INFO: `/v3/users/74cffa37b1e444308eb866bd5f883859` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.871013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c2238-33a6-4d54-ae7a-0a2d6bd675d6 None admin] RBAC: Authorizing `identity:get_user(user_id=74cffa37b1e444308eb866bd5f883859)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:32.875007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c2238-33a6-4d54-ae7a-0a2d6bd675d6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:32.882426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 140/281] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:32 2022] GET /identity/v3/users/74cffa37b1e444308eb866bd5f883859 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:32.884104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c2238-33a6-4d54-ae7a-0a2d6bd675d6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.980320 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba366883-d3ab-493c-824f-18290b12615b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.982241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:33.025861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354932a4-5f4c-4080-8f25-14b874064bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:33.026479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354932a4-5f4c-4080-8f25-14b874064bca None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:33.026765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354932a4-5f4c-4080-8f25-14b874064bca None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:33.027043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354932a4-5f4c-4080-8f25-14b874064bca None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:33.027816 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354932a4-5f4c-4080-8f25-14b874064bca None admin] RBAC: Adding query filter params (name=reader) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:33.028185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354932a4-5f4c-4080-8f25-14b874064bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:33.029720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354932a4-5f4c-4080-8f25-14b874064bca None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:33.038570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c2238-33a6-4d54-ae7a-0a2d6bd675d6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:33.041533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 142/282] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:33.047485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354932a4-5f4c-4080-8f25-14b874064bca None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:33.059827 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:33.084180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7e71d-4191-470a-8348-636383f44d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:33.088235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7e71d-4191-470a-8348-636383f44d59 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:33.089888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7e71d-4191-470a-8348-636383f44d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:33.090206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7e71d-4191-470a-8348-636383f44d59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:33.092261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7e71d-4191-470a-8348-636383f44d59 None admin] RBAC: Adding query filter params (role.id=df5932e731294f7498a6983a730628bd, user.id=74cffa37b1e444308eb866bd5f883859, scope.system=all) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:33.092830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7e71d-4191-470a-8348-636383f44d59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:33.094371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7e71d-4191-470a-8348-636383f44d59 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:33.100007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 141/283] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jul 14 11:23:32 2022] GET /identity/v3/role_assignments?user.id=74cffa37b1e444308eb866bd5f883859&scope.system=all&role.id=df5932e731294f7498a6983a730628bd => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:33.181021 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7636-7791-4723-bc41-7e388eed510c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:33.181312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7636-7791-4723-bc41-7e388eed510c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:33.181592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7636-7791-4723-bc41-7e388eed510c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:33.183013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 142/284] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:33.190662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7527b6-e898-4140-baa2-fb1ccf535b3e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:33.190662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7527b6-e898-4140-baa2-fb1ccf535b3e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:33.190662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7527b6-e898-4140-baa2-fb1ccf535b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:33.213149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354932a4-5f4c-4080-8f25-14b874064bca None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:33.218876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0a7527b6-e898-4140-baa2-fb1ccf535b3e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:33.219754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0a7527b6-e898-4140-baa2-fb1ccf535b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:33.258644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcd348-b5b2-4ff8-9f21-8e5d6ee83ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:33.259991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcd348-b5b2-4ff8-9f21-8e5d6ee83ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:33.260276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcd348-b5b2-4ff8-9f21-8e5d6ee83ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:33.260549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcd348-b5b2-4ff8-9f21-8e5d6ee83ba3 None admin] PATH_INFO: `/v3/users/e63b482b343e45eb828a1e4eb598401a` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:33.262364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcd348-b5b2-4ff8-9f21-8e5d6ee83ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=e63b482b343e45eb828a1e4eb598401a)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:33.264053 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcd348-b5b2-4ff8-9f21-8e5d6ee83ba3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:33.267159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 143/285] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:33 2022] GET /identity/v3/users/e63b482b343e45eb828a1e4eb598401a => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:33.273794 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcd348-b5b2-4ff8-9f21-8e5d6ee83ba3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:33.316557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 143/286] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:33.317884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:33.340520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7527b6-e898-4140-baa2-fb1ccf535b3e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:33.408003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcd348-b5b2-4ff8-9f21-8e5d6ee83ba3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:33.455789 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7527b6-e898-4140-baa2-fb1ccf535b3e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:33.456099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b569e-ebbf-460b-8d61-02519556c99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:33.456953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b569e-ebbf-460b-8d61-02519556c99a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:33.457529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b569e-ebbf-460b-8d61-02519556c99a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:33.457813 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b569e-ebbf-460b-8d61-02519556c99a None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:33.460043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b569e-ebbf-460b-8d61-02519556c99a None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:33.461542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b569e-ebbf-460b-8d61-02519556c99a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:33.464143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 144/287] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:33 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:33.470326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b569e-ebbf-460b-8d61-02519556c99a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:33.471120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:33.497464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8e5c9-9b96-42f2-bf8f-394dbca95fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:33.498353 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8e5c9-9b96-42f2-bf8f-394dbca95fdf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:33.498745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8e5c9-9b96-42f2-bf8f-394dbca95fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:33.499082 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8e5c9-9b96-42f2-bf8f-394dbca95fdf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:33.510534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8e5c9-9b96-42f2-bf8f-394dbca95fdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9fa8e5c9-9b96-42f2-bf8f-394dbca95fdf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:33.519199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:33.520156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:33.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 144/288] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:33.526878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8e5c9-9b96-42f2-bf8f-394dbca95fdf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:33.580289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:33.583378 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b569e-ebbf-460b-8d61-02519556c99a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:33.622384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a9995-6bbf-4313-81fa-e70bbfd9deb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:33.623479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9995-6bbf-4313-81fa-e70bbfd9deb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:33.623770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9995-6bbf-4313-81fa-e70bbfd9deb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:33.624051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9995-6bbf-4313-81fa-e70bbfd9deb3 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/users/e63b482b343e45eb828a1e4eb598401a/roles/4dc8f8c1b32c4c57805640c209616fde` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:33.626416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a9995-6bbf-4313-81fa-e70bbfd9deb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user_id=e63b482b343e45eb828a1e4eb598401a, role_id=4dc8f8c1b32c4c57805640c209616fde)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:33.627929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a9995-6bbf-4313-81fa-e70bbfd9deb3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:33.639502 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8e5c9-9b96-42f2-bf8f-394dbca95fdf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:33.676320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b019e5-4386-4c98-87af-24da08e19cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:33.676988 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b019e5-4386-4c98-87af-24da08e19cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:33.677307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b019e5-4386-4c98-87af-24da08e19cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:33.677580 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b019e5-4386-4c98-87af-24da08e19cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:33.678442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b019e5-4386-4c98-87af-24da08e19cb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:33.679277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b019e5-4386-4c98-87af-24da08e19cb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:33.680702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b019e5-4386-4c98-87af-24da08e19cb3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:33.689532 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 145/289] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:33.695002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b019e5-4386-4c98-87af-24da08e19cb3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:33.707918 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 145/290] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:23:33 2022] PUT /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/users/e63b482b343e45eb828a1e4eb598401a/roles/4dc8f8c1b32c4c57805640c209616fde => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:33.729442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:33.803386 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b019e5-4386-4c98-87af-24da08e19cb3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:33.841422 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68766edd-7a5c-4571-bc6c-bdf9f3b5c2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:33.842207 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68766edd-7a5c-4571-bc6c-bdf9f3b5c2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:33.842477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68766edd-7a5c-4571-bc6c-bdf9f3b5c2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:33.842743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68766edd-7a5c-4571-bc6c-bdf9f3b5c2e7 None admin] PATH_INFO: `/v3/users/729077e64d4a4de18a13d4ebb639e569` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:33.843932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68766edd-7a5c-4571-bc6c-bdf9f3b5c2e7 None admin] RBAC: Authorizing `identity:get_user(user_id=729077e64d4a4de18a13d4ebb639e569)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:33.845413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68766edd-7a5c-4571-bc6c-bdf9f3b5c2e7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:33.847343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 146/291] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:33 2022] GET /identity/v3/users/729077e64d4a4de18a13d4ebb639e569 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:33.853332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a9995-6bbf-4313-81fa-e70bbfd9deb3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:34.004345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a9995-6bbf-4313-81fa-e70bbfd9deb3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:34.042079 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709dc38-552e-4720-96c6-8efeb41f67fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:34.042740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709dc38-552e-4720-96c6-8efeb41f67fe None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:34.043024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709dc38-552e-4720-96c6-8efeb41f67fe None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:34.043300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709dc38-552e-4720-96c6-8efeb41f67fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:34.074317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:34.117715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709dc38-552e-4720-96c6-8efeb41f67fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:34.119173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709dc38-552e-4720-96c6-8efeb41f67fe None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:34.238087 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 146/292] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:34.243588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68766edd-7a5c-4571-bc6c-bdf9f3b5c2e7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:34.354281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68766edd-7a5c-4571-bc6c-bdf9f3b5c2e7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:34.357516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:34.403883 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5006c9-05a1-4f02-b6cd-7f375c74e224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:34.404651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5006c9-05a1-4f02-b6cd-7f375c74e224 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:34.404925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5006c9-05a1-4f02-b6cd-7f375c74e224 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:34.405215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5006c9-05a1-4f02-b6cd-7f375c74e224 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:34.406411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5006c9-05a1-4f02-b6cd-7f375c74e224 None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:34.407832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5006c9-05a1-4f02-b6cd-7f375c74e224 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:34.409555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 147/293] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:34 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:34.415445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709dc38-552e-4720-96c6-8efeb41f67fe None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:34.507193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:34.548738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709dc38-552e-4720-96c6-8efeb41f67fe None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:34.600269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8650fe8-61fb-410e-a18c-af5ec496e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:34.600942 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650fe8-61fb-410e-a18c-af5ec496e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:34.601246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650fe8-61fb-410e-a18c-af5ec496e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:34.601527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650fe8-61fb-410e-a18c-af5ec496e8f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:34.602371 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8650fe8-61fb-410e-a18c-af5ec496e8f6 None admin] RBAC: Adding query filter params (role.id=df5932e731294f7498a6983a730628bd, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user.id=729077e64d4a4de18a13d4ebb639e569) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:34.602682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8650fe8-61fb-410e-a18c-af5ec496e8f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:34.604121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8650fe8-61fb-410e-a18c-af5ec496e8f6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:34.614013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 147/294] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:34 2022] GET /identity/v3/role_assignments?user.id=729077e64d4a4de18a13d4ebb639e569&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=df5932e731294f7498a6983a730628bd => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:35.562484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8fc72-1f87-4c89-b68a-fd316a8ebaef None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:35.563540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8fc72-1f87-4c89-b68a-fd316a8ebaef None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:35.564050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8fc72-1f87-4c89-b68a-fd316a8ebaef None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:35.565894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 148/295] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:35.571630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3e7ed-ee3a-4f3a-a902-334fc9566dd8 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:35.572490 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3e7ed-ee3a-4f3a-a902-334fc9566dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:35.572969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3e7ed-ee3a-4f3a-a902-334fc9566dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:35.595817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-5bb3e7ed-ee3a-4f3a-a902-334fc9566dd8 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:35.597373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5bb3e7ed-ee3a-4f3a-a902-334fc9566dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:35.660371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cfd57-0381-40ef-8f4b-351f2457a67c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:35.660989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cfd57-0381-40ef-8f4b-351f2457a67c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:35.661525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cfd57-0381-40ef-8f4b-351f2457a67c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:35.663535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 149/296] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:35.668947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56af96-4bf9-4a55-a5ad-1ec07c373189 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:35.669713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56af96-4bf9-4a55-a5ad-1ec07c373189 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:35.670199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56af96-4bf9-4a55-a5ad-1ec07c373189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:35.677399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 148/297] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:35.688430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1d56af96-4bf9-4a55-a5ad-1ec07c373189 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:35.689980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1d56af96-4bf9-4a55-a5ad-1ec07c373189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:35.702129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3e7ed-ee3a-4f3a-a902-334fc9566dd8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:35.764070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:35.765013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 150/298] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:35.769310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e7e06-19cf-4645-a29f-2303a1d28adb None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:35.769310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e7e06-19cf-4645-a29f-2303a1d28adb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:35.769310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e7e06-19cf-4645-a29f-2303a1d28adb None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:35.769310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 151/299] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:35.774459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a1d86-7a97-4a76-afdc-b2960e5a9cb7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:35.774924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a1d86-7a97-4a76-afdc-b2960e5a9cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:35.775334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a1d86-7a97-4a76-afdc-b2960e5a9cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:35.793534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-434a1d86-7a97-4a76-afdc-b2960e5a9cb7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:35.794943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-434a1d86-7a97-4a76-afdc-b2960e5a9cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:35.821361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3e7ed-ee3a-4f3a-a902-334fc9566dd8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:35.869621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfffa37-fc27-4a0a-b03b-ae0213501f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:35.870432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfffa37-fc27-4a0a-b03b-ae0213501f1c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:35.881331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfffa37-fc27-4a0a-b03b-ae0213501f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:35.881480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 152/300] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:35.881582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfffa37-fc27-4a0a-b03b-ae0213501f1c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:35.881656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a1d86-7a97-4a76-afdc-b2960e5a9cb7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:35.893992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfffa37-fc27-4a0a-b03b-ae0213501f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-adfffa37-fc27-4a0a-b03b-ae0213501f1c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:35.902132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:35.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 149/301] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:35.904976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfffa37-fc27-4a0a-b03b-ae0213501f1c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:35.923462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:35.997903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a1d86-7a97-4a76-afdc-b2960e5a9cb7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:36.028462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfffa37-fc27-4a0a-b03b-ae0213501f1c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:36.034637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ca0a2-5766-4d3f-ad07-107dfb478f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:36.035438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ca0a2-5766-4d3f-ad07-107dfb478f53 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.035720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ca0a2-5766-4d3f-ad07-107dfb478f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.035993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ca0a2-5766-4d3f-ad07-107dfb478f53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.037622 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ca0a2-5766-4d3f-ad07-107dfb478f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:36.040034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ca0a2-5766-4d3f-ad07-107dfb478f53 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:36.042034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 153/302] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:36.042807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ca0a2-5766-4d3f-ad07-107dfb478f53 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:36.048345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:36.070782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2639972-4906-4296-afa7-a06297b4753f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:36.071635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2639972-4906-4296-afa7-a06297b4753f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.072089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2639972-4906-4296-afa7-a06297b4753f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.072365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2639972-4906-4296-afa7-a06297b4753f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.084312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2639972-4906-4296-afa7-a06297b4753f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-a2639972-4906-4296-afa7-a06297b4753f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:36.092412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:36.093443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:36.094236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:36.096792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 150/303] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:36.098408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2639972-4906-4296-afa7-a06297b4753f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:36.148004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:36.153867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ca0a2-5766-4d3f-ad07-107dfb478f53 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:36.192551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab7a61-7806-4e35-ae37-0f3cd1acdcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:36.193244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab7a61-7806-4e35-ae37-0f3cd1acdcff None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.193525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab7a61-7806-4e35-ae37-0f3cd1acdcff None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.193794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab7a61-7806-4e35-ae37-0f3cd1acdcff None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.195099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab7a61-7806-4e35-ae37-0f3cd1acdcff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:36.195403 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab7a61-7806-4e35-ae37-0f3cd1acdcff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:36.196782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab7a61-7806-4e35-ae37-0f3cd1acdcff None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:36.205739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 154/304] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:36.206554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab7a61-7806-4e35-ae37-0f3cd1acdcff None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:36.213065 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2639972-4906-4296-afa7-a06297b4753f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:36.250248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de40e7-317f-4027-b622-657ef8ddfa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:36.250909 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de40e7-317f-4027-b622-657ef8ddfa42 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.251189 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de40e7-317f-4027-b622-657ef8ddfa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.251461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de40e7-317f-4027-b622-657ef8ddfa42 None admin] PATH_INFO: `/v3/groups` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.252158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de40e7-317f-4027-b622-657ef8ddfa42 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:36.253656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de40e7-317f-4027-b622-657ef8ddfa42 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:36.311317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab7a61-7806-4e35-ae37-0f3cd1acdcff None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:36.343969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d14a6b-2293-4678-93ac-113cdcee8a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:36.344556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14a6b-2293-4678-93ac-113cdcee8a69 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.344834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14a6b-2293-4678-93ac-113cdcee8a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.345104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14a6b-2293-4678-93ac-113cdcee8a69 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.345873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14a6b-2293-4678-93ac-113cdcee8a69 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:36.346173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14a6b-2293-4678-93ac-113cdcee8a69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:36.347533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14a6b-2293-4678-93ac-113cdcee8a69 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:36.355773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 155/305] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:36.356547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d14a6b-2293-4678-93ac-113cdcee8a69 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:36.378111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:36.389898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 151/306] 10.222.0.40 () {60 vars in 1134 bytes} [Thu Jul 14 11:23:36 2022] POST /identity/v3/groups => generated 229 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:36.394800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba5ba3-78ff-43c6-8cd6-015f3da3065f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.394800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba5ba3-78ff-43c6-8cd6-015f3da3065f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.394800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba5ba3-78ff-43c6-8cd6-015f3da3065f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.394800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 152/307] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:36.394800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba5ba3-78ff-43c6-8cd6-015f3da3065f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:36.448075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:36.472454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d14a6b-2293-4678-93ac-113cdcee8a69 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:36.511655 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116f1e4-a57c-4b10-ae70-8b6f725eecca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:36.512714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116f1e4-a57c-4b10-ae70-8b6f725eecca None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.513076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116f1e4-a57c-4b10-ae70-8b6f725eecca None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.513455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116f1e4-a57c-4b10-ae70-8b6f725eecca None admin] PATH_INFO: `/v3/users/d765d8f918cd4155bb0209ba9e4ee9ad` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.520681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba5ba3-78ff-43c6-8cd6-015f3da3065f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:36.529739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116f1e4-a57c-4b10-ae70-8b6f725eecca None admin] RBAC: Authorizing `identity:get_user(user_id=d765d8f918cd4155bb0209ba9e4ee9ad)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:36.531302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116f1e4-a57c-4b10-ae70-8b6f725eecca None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:36.533096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 156/308] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:36 2022] GET /identity/v3/users/d765d8f918cd4155bb0209ba9e4ee9ad => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:36.534443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a4b30-a878-4308-93c5-a1ad816e84a5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.537969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a4b30-a878-4308-93c5-a1ad816e84a5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.537969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a4b30-a878-4308-93c5-a1ad816e84a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.554606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9d9a4b30-a878-4308-93c5-a1ad816e84a5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:36.555808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9d9a4b30-a878-4308-93c5-a1ad816e84a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:36.573020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef2349-3536-40ab-9b50-fb46f2a7c887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:36.573940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2349-3536-40ab-9b50-fb46f2a7c887 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.574237 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2349-3536-40ab-9b50-fb46f2a7c887 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.574512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2349-3536-40ab-9b50-fb46f2a7c887 None admin] PATH_INFO: `/v3/users/74cffa37b1e444308eb866bd5f883859` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.576648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef2349-3536-40ab-9b50-fb46f2a7c887 None admin] RBAC: Authorizing `identity:get_user(user_id=74cffa37b1e444308eb866bd5f883859)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:36.578989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef2349-3536-40ab-9b50-fb46f2a7c887 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:36.581693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 153/309] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:36 2022] GET /identity/v3/users/74cffa37b1e444308eb866bd5f883859 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:36.581693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef2349-3536-40ab-9b50-fb46f2a7c887 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:36.668421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 157/310] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:36.669812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a4b30-a878-4308-93c5-a1ad816e84a5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:36.693195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:36.702297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef2349-3536-40ab-9b50-fb46f2a7c887 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:36.704555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:36.738451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0dcef-93d6-4603-881a-28fdb814e52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:36.739135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0dcef-93d6-4603-881a-28fdb814e52f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.739416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0dcef-93d6-4603-881a-28fdb814e52f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.739692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0dcef-93d6-4603-881a-28fdb814e52f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.740578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0dcef-93d6-4603-881a-28fdb814e52f None admin] RBAC: Adding query filter params (role.id=4dc8f8c1b32c4c57805640c209616fde, user.id=d765d8f918cd4155bb0209ba9e4ee9ad, scope.system=all) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:36.740926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0dcef-93d6-4603-881a-28fdb814e52f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:36.742400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0dcef-93d6-4603-881a-28fdb814e52f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:36.747966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 154/311] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jul 14 11:23:36 2022] GET /identity/v3/role_assignments?user.id=d765d8f918cd4155bb0209ba9e4ee9ad&scope.system=all&role.id=4dc8f8c1b32c4c57805640c209616fde => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:36.748542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0dcef-93d6-4603-881a-28fdb814e52f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:36.836412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a4b30-a878-4308-93c5-a1ad816e84a5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:36.881146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0dcef-93d6-4603-881a-28fdb814e52f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:36.889221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad047435-03c2-4132-abae-36e801494db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:36.890627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad047435-03c2-4132-abae-36e801494db1 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.891098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad047435-03c2-4132-abae-36e801494db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.891555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad047435-03c2-4132-abae-36e801494db1 None admin] PATH_INFO: `/v3/system/users/74cffa37b1e444308eb866bd5f883859/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.893417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad047435-03c2-4132-abae-36e801494db1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=74cffa37b1e444308eb866bd5f883859, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:36.895088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad047435-03c2-4132-abae-36e801494db1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:36.921233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cb4e7-30ed-4130-bc2d-ac3ff2b65ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:36.922051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb4e7-30ed-4130-bc2d-ac3ff2b65ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:36.922399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb4e7-30ed-4130-bc2d-ac3ff2b65ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:36.922740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb4e7-30ed-4130-bc2d-ac3ff2b65ef1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:36.935493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13cb4e7-30ed-4130-bc2d-ac3ff2b65ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-b13cb4e7-30ed-4130-bc2d-ac3ff2b65ef1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:36.943493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:36.945031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:36.946569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 155/312] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:36.951410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cb4e7-30ed-4130-bc2d-ac3ff2b65ef1 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:36.982236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:37.005043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 158/313] 10.222.0.40 () {58 vars in 1334 bytes} [Thu Jul 14 11:23:36 2022] PUT /identity/v3/system/users/74cffa37b1e444308eb866bd5f883859/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:37.071882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cb4e7-30ed-4130-bc2d-ac3ff2b65ef1 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:37.111529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8c80d-2c0e-4d9e-9051-6014cee0a43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:37.112272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8c80d-2c0e-4d9e-9051-6014cee0a43e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:37.112608 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8c80d-2c0e-4d9e-9051-6014cee0a43e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:37.112885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8c80d-2c0e-4d9e-9051-6014cee0a43e None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:37.113684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8c80d-2c0e-4d9e-9051-6014cee0a43e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:37.113989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8c80d-2c0e-4d9e-9051-6014cee0a43e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:37.115464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8c80d-2c0e-4d9e-9051-6014cee0a43e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:37.129198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 156/314] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:37.138570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad047435-03c2-4132-abae-36e801494db1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:37.217368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:37.249637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad047435-03c2-4132-abae-36e801494db1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:37.286306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d7908-25cd-44fd-96c6-1e23b14eb874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:37.287424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d7908-25cd-44fd-96c6-1e23b14eb874 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:37.288379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d7908-25cd-44fd-96c6-1e23b14eb874 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:37.288821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d7908-25cd-44fd-96c6-1e23b14eb874 None admin] PATH_INFO: `/v3/users/e63b482b343e45eb828a1e4eb598401a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:37.290238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d7908-25cd-44fd-96c6-1e23b14eb874 None admin] RBAC: Authorizing `identity:get_user(user_id=e63b482b343e45eb828a1e4eb598401a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:37.291843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d7908-25cd-44fd-96c6-1e23b14eb874 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:37.298375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 159/315] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:37 2022] GET /identity/v3/users/e63b482b343e45eb828a1e4eb598401a => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:37.299945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8c80d-2c0e-4d9e-9051-6014cee0a43e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:37.397993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:37.410596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8c80d-2c0e-4d9e-9051-6014cee0a43e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:37.447679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6215858-07e7-418a-a173-7a87cb732c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:37.450979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6215858-07e7-418a-a173-7a87cb732c40 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:37.450979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6215858-07e7-418a-a173-7a87cb732c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:37.450979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6215858-07e7-418a-a173-7a87cb732c40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:37.521141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6215858-07e7-418a-a173-7a87cb732c40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:37.522728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6215858-07e7-418a-a173-7a87cb732c40 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:37.617488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:37.630833 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 157/316] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:37.636413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d7908-25cd-44fd-96c6-1e23b14eb874 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:37.743927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d7908-25cd-44fd-96c6-1e23b14eb874 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:37.778387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb3973-9e51-4ae4-8c22-72846dec13d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:37.779173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb3973-9e51-4ae4-8c22-72846dec13d3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:37.779754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb3973-9e51-4ae4-8c22-72846dec13d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:37.780031 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb3973-9e51-4ae4-8c22-72846dec13d3 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:37.781266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb3973-9e51-4ae4-8c22-72846dec13d3 None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:37.782722 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb3973-9e51-4ae4-8c22-72846dec13d3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:37.784739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 160/317] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:37 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:37.790833 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6215858-07e7-418a-a173-7a87cb732c40 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:37.901620 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6215858-07e7-418a-a173-7a87cb732c40 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:37.938050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ca2f3-280b-4a21-8f7a-694763fa04e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:37.939100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ca2f3-280b-4a21-8f7a-694763fa04e5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:37.939600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ca2f3-280b-4a21-8f7a-694763fa04e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:37.940105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ca2f3-280b-4a21-8f7a-694763fa04e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:37.941239 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ca2f3-280b-4a21-8f7a-694763fa04e5 None admin] RBAC: Adding query filter params (role.id=4dc8f8c1b32c4c57805640c209616fde, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9, user.id=e63b482b343e45eb828a1e4eb598401a) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:37.941782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ca2f3-280b-4a21-8f7a-694763fa04e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:37.943485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ca2f3-280b-4a21-8f7a-694763fa04e5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:37.950887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 158/318] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:23:37 2022] GET /identity/v3/role_assignments?user.id=e63b482b343e45eb828a1e4eb598401a&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=4dc8f8c1b32c4c57805640c209616fde => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:38.609812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16ae2e-25d5-4bde-945d-b6e1faab26eb None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:38.610223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16ae2e-25d5-4bde-945d-b6e1faab26eb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:38.610401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16ae2e-25d5-4bde-945d-b6e1faab26eb None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:38.611926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 161/319] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:38.617054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75c0f2-388d-422f-9e5b-e4621252942e None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:38.617371 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75c0f2-388d-422f-9e5b-e4621252942e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:38.617653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75c0f2-388d-422f-9e5b-e4621252942e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:38.634466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-5f75c0f2-388d-422f-9e5b-e4621252942e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:38.635674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5f75c0f2-388d-422f-9e5b-e4621252942e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:38.687319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:38.708140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 159/320] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:38.731213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16ae2e-25d5-4bde-945d-b6e1faab26eb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:38.742681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:38.842085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16ae2e-25d5-4bde-945d-b6e1faab26eb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:38.860917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327524a1-bb98-425c-91de-da435143df4f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:38.861285 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327524a1-bb98-425c-91de-da435143df4f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:38.861443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327524a1-bb98-425c-91de-da435143df4f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:38.863049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 160/321] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:23:38.870580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1d3e-1677-4aab-97ce-c38719093281 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:38.870930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1d3e-1677-4aab-97ce-c38719093281 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:38.871219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1d3e-1677-4aab-97ce-c38719093281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:38.888618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ea4f-7445-408e-85d0-2d599106fabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:38.889526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ea4f-7445-408e-85d0-2d599106fabe None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:38.889743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ea4f-7445-408e-85d0-2d599106fabe None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:38.889971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4ea4f-7445-408e-85d0-2d599106fabe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:38.891136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4ea4f-7445-408e-85d0-2d599106fabe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:38.892563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4ea4f-7445-408e-85d0-2d599106fabe None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:38.892665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-6c2b1d3e-1677-4aab-97ce-c38719093281 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:38.893999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6c2b1d3e-1677-4aab-97ce-c38719093281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:38.894544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 162/322] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:38.899819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ea4f-7445-408e-85d0-2d599106fabe None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:38.970959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 161/323] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:38.985825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:38.993765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b1d3e-1677-4aab-97ce-c38719093281 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:39.009517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4ea4f-7445-408e-85d0-2d599106fabe None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:39.045847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225c83a-00b8-4c4e-89a9-1dacfb596798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:39.046521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225c83a-00b8-4c4e-89a9-1dacfb596798 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:39.046806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225c83a-00b8-4c4e-89a9-1dacfb596798 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:39.047084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225c83a-00b8-4c4e-89a9-1dacfb596798 None admin] PATH_INFO: `/v3/groups` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:39.047783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225c83a-00b8-4c4e-89a9-1dacfb596798 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:39.049248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225c83a-00b8-4c4e-89a9-1dacfb596798 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:39.095250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b1d3e-1677-4aab-97ce-c38719093281 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:39.108333 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:39.128941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed1e36-7ac8-4a86-8e55-0b5748e83c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:39.129698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed1e36-7ac8-4a86-8e55-0b5748e83c69 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:39.129978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed1e36-7ac8-4a86-8e55-0b5748e83c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:39.130251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed1e36-7ac8-4a86-8e55-0b5748e83c69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:39.142046 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed1e36-7ac8-4a86-8e55-0b5748e83c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:39.150371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 163/324] 10.222.0.40 () {60 vars in 1134 bytes} [Thu Jul 14 11:23:38 2022] POST /identity/v3/groups => generated 226 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-36ed1e36-7ac8-4a86-8e55-0b5748e83c69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:39.151281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:39.155668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:39.156511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:39.157129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02ec2d-88bd-4429-8a23-10b59522b4cc None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:39.157129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02ec2d-88bd-4429-8a23-10b59522b4cc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:39.157129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02ec2d-88bd-4429-8a23-10b59522b4cc None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:39.157507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 162/325] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:23:39.157640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 164/326] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:39.159918 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637c11e-381c-401d-b59c-7255743415aa None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:39.160172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637c11e-381c-401d-b59c-7255743415aa None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:39.160455 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637c11e-381c-401d-b59c-7255743415aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:39.161423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02ec2d-88bd-4429-8a23-10b59522b4cc None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:39.177958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-3637c11e-381c-401d-b59c-7255743415aa None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:39.179734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3637c11e-381c-401d-b59c-7255743415aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:39.259039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 163/327] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:39.282405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637c11e-381c-401d-b59c-7255743415aa None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:39.288942 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02ec2d-88bd-4429-8a23-10b59522b4cc None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:39.326716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cbe4a-c0d8-4352-b94e-9cb5209d279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:39.327761 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cbe4a-c0d8-4352-b94e-9cb5209d279d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:39.328127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cbe4a-c0d8-4352-b94e-9cb5209d279d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:39.328405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cbe4a-c0d8-4352-b94e-9cb5209d279d None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:39.329249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cbe4a-c0d8-4352-b94e-9cb5209d279d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:39.329494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cbe4a-c0d8-4352-b94e-9cb5209d279d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:39.330907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cbe4a-c0d8-4352-b94e-9cb5209d279d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:39.334231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:39.342673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 165/328] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:39.347897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cbe4a-c0d8-4352-b94e-9cb5209d279d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:39.414150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637c11e-381c-401d-b59c-7255743415aa None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:39.414729 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:39.459496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41616bed-f123-41d3-a79d-29480d8c86ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:39.460305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41616bed-f123-41d3-a79d-29480d8c86ee None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:39.460585 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41616bed-f123-41d3-a79d-29480d8c86ee None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:39.460972 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41616bed-f123-41d3-a79d-29480d8c86ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:39.472229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cbe4a-c0d8-4352-b94e-9cb5209d279d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:39.476607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41616bed-f123-41d3-a79d-29480d8c86ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-41616bed-f123-41d3-a79d-29480d8c86ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:39.485382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:39.486481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:39.487387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:39.488022 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 164/329] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:23:39.493631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41616bed-f123-41d3-a79d-29480d8c86ee None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:39.516400 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc40bb-08a3-47aa-947c-42062d51b749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:39.517321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc40bb-08a3-47aa-947c-42062d51b749 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:39.517601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc40bb-08a3-47aa-947c-42062d51b749 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:39.517873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc40bb-08a3-47aa-947c-42062d51b749 None admin] PATH_INFO: `/v3/users/d765d8f918cd4155bb0209ba9e4ee9ad` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:39.519303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc40bb-08a3-47aa-947c-42062d51b749 None admin] RBAC: Authorizing `identity:get_user(user_id=d765d8f918cd4155bb0209ba9e4ee9ad)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:39.520948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc40bb-08a3-47aa-947c-42062d51b749 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:39.528227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 166/330] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:39 2022] GET /identity/v3/users/d765d8f918cd4155bb0209ba9e4ee9ad => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:39.530697 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc40bb-08a3-47aa-947c-42062d51b749 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:39.615184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:39.631921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41616bed-f123-41d3-a79d-29480d8c86ee None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:39.663619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc40bb-08a3-47aa-947c-42062d51b749 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:39.671159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc193c-0cbb-4af1-91b1-ae9156196d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:39.671755 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc193c-0cbb-4af1-91b1-ae9156196d7f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:39.672036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc193c-0cbb-4af1-91b1-ae9156196d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:39.672311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc193c-0cbb-4af1-91b1-ae9156196d7f None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:39.673076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc193c-0cbb-4af1-91b1-ae9156196d7f None admin] RBAC: Adding query filter params (name=member) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:39.673527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc193c-0cbb-4af1-91b1-ae9156196d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:39.675077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc193c-0cbb-4af1-91b1-ae9156196d7f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:39.689793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 165/331] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:39.696050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc193c-0cbb-4af1-91b1-ae9156196d7f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:39.712893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f56ffb-eb64-4e82-830d-605b05939def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:39.714320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f56ffb-eb64-4e82-830d-605b05939def None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:39.714874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f56ffb-eb64-4e82-830d-605b05939def None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:39.715429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f56ffb-eb64-4e82-830d-605b05939def None admin] PATH_INFO: `/v3/system/users/d765d8f918cd4155bb0209ba9e4ee9ad/roles/4dc8f8c1b32c4c57805640c209616fde` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:39.717882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f56ffb-eb64-4e82-830d-605b05939def None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d765d8f918cd4155bb0209ba9e4ee9ad, role_id=4dc8f8c1b32c4c57805640c209616fde)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:39.720828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f56ffb-eb64-4e82-830d-605b05939def None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:39.727845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:39.818021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc193c-0cbb-4af1-91b1-ae9156196d7f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:39.818566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 167/332] 10.222.0.40 () {58 vars in 1334 bytes} [Thu Jul 14 11:23:39 2022] PUT /identity/v3/system/users/d765d8f918cd4155bb0209ba9e4ee9ad/roles/4dc8f8c1b32c4c57805640c209616fde => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:39.859449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f401a-4657-4f88-9df2-810f929574b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:39.860586 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f401a-4657-4f88-9df2-810f929574b2 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:39.861051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f401a-4657-4f88-9df2-810f929574b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:39.862049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f401a-4657-4f88-9df2-810f929574b2 None admin] PATH_INFO: `/v3/users/74cffa37b1e444308eb866bd5f883859` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:39.867596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f401a-4657-4f88-9df2-810f929574b2 None admin] RBAC: Authorizing `identity:get_user(user_id=74cffa37b1e444308eb866bd5f883859)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:39.869470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f401a-4657-4f88-9df2-810f929574b2 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:39.873849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 166/333] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:39 2022] GET /identity/v3/users/74cffa37b1e444308eb866bd5f883859 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:39.880276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f56ffb-eb64-4e82-830d-605b05939def None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:40.009569 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f56ffb-eb64-4e82-830d-605b05939def None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:40.043866 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:40.050762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095921b8-c1c7-407b-b7b1-26707f560c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:40.051429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095921b8-c1c7-407b-b7b1-26707f560c43 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:40.051714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095921b8-c1c7-407b-b7b1-26707f560c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:40.051991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095921b8-c1c7-407b-b7b1-26707f560c43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:40.052836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095921b8-c1c7-407b-b7b1-26707f560c43 None admin] RBAC: Adding query filter params (role.id=df5932e731294f7498a6983a730628bd, user.id=74cffa37b1e444308eb866bd5f883859, scope.system=all) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:40.053145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095921b8-c1c7-407b-b7b1-26707f560c43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:40.054684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095921b8-c1c7-407b-b7b1-26707f560c43 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:40.067983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 168/334] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jul 14 11:23:39 2022] GET /identity/v3/role_assignments?user.id=74cffa37b1e444308eb866bd5f883859&scope.system=all&role.id=df5932e731294f7498a6983a730628bd => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:41.633344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff090-6b26-47a3-b43f-43b618e0e442 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:41.633903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff090-6b26-47a3-b43f-43b618e0e442 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:41.633903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff090-6b26-47a3-b43f-43b618e0e442 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:41.635319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 167/335] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:41.641055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45cdce-40f8-4f48-b173-f0e93da406c2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:41.641506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45cdce-40f8-4f48-b173-f0e93da406c2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:41.641700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45cdce-40f8-4f48-b173-f0e93da406c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:41.666181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ad45cdce-40f8-4f48-b173-f0e93da406c2 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:41.667395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ad45cdce-40f8-4f48-b173-f0e93da406c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:41.752902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 169/336] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:41.775394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff090-6b26-47a3-b43f-43b618e0e442 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:41.893749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff090-6b26-47a3-b43f-43b618e0e442 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:41.896260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:41.932470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37a9e1-d87a-4333-a541-2aa0a3c740a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:41.933317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37a9e1-d87a-4333-a541-2aa0a3c740a8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:41.933598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37a9e1-d87a-4333-a541-2aa0a3c740a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:41.933870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37a9e1-d87a-4333-a541-2aa0a3c740a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:41.945555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37a9e1-d87a-4333-a541-2aa0a3c740a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-8a37a9e1-d87a-4333-a541-2aa0a3c740a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:41.954334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:41.955240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:41.956054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:41.960824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 168/337] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:41.962279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45cdce-40f8-4f48-b173-f0e93da406c2 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:42.087251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45cdce-40f8-4f48-b173-f0e93da406c2 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:42.114619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:42.136590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab579f2-a81e-482d-9d15-8a6490a6e535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:42.137306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab579f2-a81e-482d-9d15-8a6490a6e535 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:42.137588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab579f2-a81e-482d-9d15-8a6490a6e535 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:42.137859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab579f2-a81e-482d-9d15-8a6490a6e535 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:42.138620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab579f2-a81e-482d-9d15-8a6490a6e535 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:42.138921 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab579f2-a81e-482d-9d15-8a6490a6e535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:42.140300 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab579f2-a81e-482d-9d15-8a6490a6e535 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:42.149855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 170/338] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:42.155677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37a9e1-d87a-4333-a541-2aa0a3c740a8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:42.274887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa140a-9f67-42ef-9736-b7251f2a68cc None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:42.275234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa140a-9f67-42ef-9736-b7251f2a68cc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:42.275445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa140a-9f67-42ef-9736-b7251f2a68cc None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:42.276968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 171/339] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:23:42.286824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529783a-700e-448c-a24e-e9779508a7e1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:42.287107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529783a-700e-448c-a24e-e9779508a7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:42.287418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529783a-700e-448c-a24e-e9779508a7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:42.313765 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e529783a-700e-448c-a24e-e9779508a7e1 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:42.315021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37a9e1-d87a-4333-a541-2aa0a3c740a8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:42.322387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e529783a-700e-448c-a24e-e9779508a7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:42.394012 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ee484-2819-4e9c-996a-ad77d029baee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:42.395117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee484-2819-4e9c-996a-ad77d029baee None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:42.395610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee484-2819-4e9c-996a-ad77d029baee None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:42.396089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee484-2819-4e9c-996a-ad77d029baee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:42.434621 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 172/340] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:42.436198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eeff8-c275-4eab-8dd4-4b7c771ed270 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:42.436487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eeff8-c275-4eab-8dd4-4b7c771ed270 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:42.436764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eeff8-c275-4eab-8dd4-4b7c771ed270 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:42.438767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 173/341] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:42.446000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e05b8d-6c0e-42ce-8844-f87070c2ea4c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:42.446286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e05b8d-6c0e-42ce-8844-f87070c2ea4c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:42.446562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e05b8d-6c0e-42ce-8844-f87070c2ea4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:42.448629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:42.482936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c1e05b8d-6c0e-42ce-8844-f87070c2ea4c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:42.484136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c1e05b8d-6c0e-42ce-8844-f87070c2ea4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:42.544981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ee484-2819-4e9c-996a-ad77d029baee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:42.546477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ee484-2819-4e9c-996a-ad77d029baee None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:42.591320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 174/342] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:42.592815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e05b8d-6c0e-42ce-8844-f87070c2ea4c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:42.645752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:42.724048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e05b8d-6c0e-42ce-8844-f87070c2ea4c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:42.771666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 169/343] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:42.776027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ee484-2819-4e9c-996a-ad77d029baee None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:42.795589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f896a-00dd-4513-a15b-62b77573abea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:42.796709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f896a-00dd-4513-a15b-62b77573abea None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:42.797420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f896a-00dd-4513-a15b-62b77573abea None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:42.798460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f896a-00dd-4513-a15b-62b77573abea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:42.817952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f896a-00dd-4513-a15b-62b77573abea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-957f896a-00dd-4513-a15b-62b77573abea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:42.827707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:42.828792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:42.829633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:42.832592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 175/344] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:42.833863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f896a-00dd-4513-a15b-62b77573abea None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:42.875293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:42.940760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ee484-2819-4e9c-996a-ad77d029baee None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:42.994720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9e0fe-2a03-4e79-9f92-f70de7f54634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:42.995453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9e0fe-2a03-4e79-9f92-f70de7f54634 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:42.995731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9e0fe-2a03-4e79-9f92-f70de7f54634 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:42.996000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9e0fe-2a03-4e79-9f92-f70de7f54634 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:43.003753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:43.010869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f896a-00dd-4513-a15b-62b77573abea None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:43.018322 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9e0fe-2a03-4e79-9f92-f70de7f54634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-afd9e0fe-2a03-4e79-9f92-f70de7f54634 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:43.031010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:43.031976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:43.033383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:43.038161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 170/345] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:43.040760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b839f-3be8-4205-ba0b-9d6b4419f4c7 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:43.041529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b839f-3be8-4205-ba0b-9d6b4419f4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:43.042005 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b839f-3be8-4205-ba0b-9d6b4419f4c7 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:43.044771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 171/346] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:43.045985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b839f-3be8-4205-ba0b-9d6b4419f4c7 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:43.056328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ca816-1525-4dd8-a7e8-b169b5694e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:43.057637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ca816-1525-4dd8-a7e8-b169b5694e69 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:43.058141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ca816-1525-4dd8-a7e8-b169b5694e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:43.059054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ca816-1525-4dd8-a7e8-b169b5694e69 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:43.060859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ca816-1525-4dd8-a7e8-b169b5694e69 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:43.062948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ca816-1525-4dd8-a7e8-b169b5694e69 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:43.069565 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 176/347] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:42 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:43.070409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ca816-1525-4dd8-a7e8-b169b5694e69 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:43.189982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b839f-3be8-4205-ba0b-9d6b4419f4c7 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:43.211311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ca816-1525-4dd8-a7e8-b169b5694e69 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:43.216772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:43.246660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3103e-79ca-49ae-b26e-a5cfaf4d1ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:43.247760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3103e-79ca-49ae-b26e-a5cfaf4d1ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:43.248043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3103e-79ca-49ae-b26e-a5cfaf4d1ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:43.248398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3103e-79ca-49ae-b26e-a5cfaf4d1ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:43.249183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3103e-79ca-49ae-b26e-a5cfaf4d1ff6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:43.249517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3103e-79ca-49ae-b26e-a5cfaf4d1ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:43.251393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3103e-79ca-49ae-b26e-a5cfaf4d1ff6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:43.261977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f6a6f-246f-47a8-b7bd-3e5c15ce2d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:43.262866 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f6a6f-246f-47a8-b7bd-3e5c15ce2d36 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:43.263292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f6a6f-246f-47a8-b7bd-3e5c15ce2d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:43.263535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 172/348] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:43.263930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f6a6f-246f-47a8-b7bd-3e5c15ce2d36 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:43.265036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f6a6f-246f-47a8-b7bd-3e5c15ce2d36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:43.265485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa8585-122d-42fd-bf0d-007f6229e946 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:43.265644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f6a6f-246f-47a8-b7bd-3e5c15ce2d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:43.266181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa8585-122d-42fd-bf0d-007f6229e946 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:43.266842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa8585-122d-42fd-bf0d-007f6229e946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:43.267610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f6a6f-246f-47a8-b7bd-3e5c15ce2d36 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:43.278453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 177/349] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:43.279698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f6a6f-246f-47a8-b7bd-3e5c15ce2d36 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:43.295339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-87aa8585-122d-42fd-bf0d-007f6229e946 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:43.299305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-87aa8585-122d-42fd-bf0d-007f6229e946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:43.320611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:43.391963 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 173/350] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:43.393261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa8585-122d-42fd-bf0d-007f6229e946 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:43.425532 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f6a6f-246f-47a8-b7bd-3e5c15ce2d36 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:43.475416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e7daa-d12a-42e8-8053-a1b2eeee9bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:43.476320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e7daa-d12a-42e8-8053-a1b2eeee9bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:43.476626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e7daa-d12a-42e8-8053-a1b2eeee9bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:43.476894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e7daa-d12a-42e8-8053-a1b2eeee9bb6 None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:43.484191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e7daa-d12a-42e8-8053-a1b2eeee9bb6 None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:43.485751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e7daa-d12a-42e8-8053-a1b2eeee9bb6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:43.487751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 178/351] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:43 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:43.488611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e7daa-d12a-42e8-8053-a1b2eeee9bb6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:43.531293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa8585-122d-42fd-bf0d-007f6229e946 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:43.534999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:43.588500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff01cb5-2fc0-440f-a002-0c40bf75da90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:43.589208 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff01cb5-2fc0-440f-a002-0c40bf75da90 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:43.589505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff01cb5-2fc0-440f-a002-0c40bf75da90 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:43.589781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff01cb5-2fc0-440f-a002-0c40bf75da90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:43.618188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:43.628939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e7daa-d12a-42e8-8053-a1b2eeee9bb6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:43.687561 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240920f7-01d0-4a87-992d-45cd37e76454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:43.688246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240920f7-01d0-4a87-992d-45cd37e76454 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:43.688534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240920f7-01d0-4a87-992d-45cd37e76454 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:43.688809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240920f7-01d0-4a87-992d-45cd37e76454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:43.696650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff01cb5-2fc0-440f-a002-0c40bf75da90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:43.698554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff01cb5-2fc0-440f-a002-0c40bf75da90 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:43.790410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240920f7-01d0-4a87-992d-45cd37e76454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:43.792120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240920f7-01d0-4a87-992d-45cd37e76454 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:43.832332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 174/352] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:43.834195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff01cb5-2fc0-440f-a002-0c40bf75da90 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:43.837324 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:43.977777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:43.984640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff01cb5-2fc0-440f-a002-0c40bf75da90 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:43.988725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 179/353] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:43.989589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240920f7-01d0-4a87-992d-45cd37e76454 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:44.023477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3368d9-2ac0-423a-be50-d49262a437e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:44.024390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3368d9-2ac0-423a-be50-d49262a437e6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.024787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3368d9-2ac0-423a-be50-d49262a437e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.025067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3368d9-2ac0-423a-be50-d49262a437e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.038376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3368d9-2ac0-423a-be50-d49262a437e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-2c3368d9-2ac0-423a-be50-d49262a437e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:44.046605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:44.047488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:44.048293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 175/354] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:44.049458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dff2c-d68f-4614-ba0c-e2f061c52332 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.049458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dff2c-d68f-4614-ba0c-e2f061c52332 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.049620 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dff2c-d68f-4614-ba0c-e2f061c52332 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.050967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 176/355] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:44.051518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dff2c-d68f-4614-ba0c-e2f061c52332 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:44.125544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240920f7-01d0-4a87-992d-45cd37e76454 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:44.138685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:44.168348 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51700cca-bd38-4881-9b91-2caffedf7189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:44.169015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51700cca-bd38-4881-9b91-2caffedf7189 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.169453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51700cca-bd38-4881-9b91-2caffedf7189 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.169779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51700cca-bd38-4881-9b91-2caffedf7189 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.170619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51700cca-bd38-4881-9b91-2caffedf7189 None admin] RBAC: Adding query filter params (group.id=8aef3e683eb24cba992d6f7b8d076340, role.id=df5932e731294f7498a6983a730628bd, scope.project.id=d76703b19c6c4521a9480e46863c1979) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:44.170931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51700cca-bd38-4881-9b91-2caffedf7189 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:44.172372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51700cca-bd38-4881-9b91-2caffedf7189 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:44.178791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 180/356] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jul 14 11:23:43 2022] GET /identity/v3/role_assignments?group.id=8aef3e683eb24cba992d6f7b8d076340&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=df5932e731294f7498a6983a730628bd => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:44.179602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51700cca-bd38-4881-9b91-2caffedf7189 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:44.189371 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dff2c-d68f-4614-ba0c-e2f061c52332 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:44.245427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da515026-2578-43a7-81f7-b73fcbd76823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:44.246763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da515026-2578-43a7-81f7-b73fcbd76823 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.247127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da515026-2578-43a7-81f7-b73fcbd76823 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.247475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da515026-2578-43a7-81f7-b73fcbd76823 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.248900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da515026-2578-43a7-81f7-b73fcbd76823 None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:44.250652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da515026-2578-43a7-81f7-b73fcbd76823 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:44.252556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 177/357] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:44 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:44.253481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da515026-2578-43a7-81f7-b73fcbd76823 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:44.258615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:44.328603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51700cca-bd38-4881-9b91-2caffedf7189 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:44.382428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36caf94-ecf4-4b21-88d7-a4165a2ba106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:44.383291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36caf94-ecf4-4b21-88d7-a4165a2ba106 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.383571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36caf94-ecf4-4b21-88d7-a4165a2ba106 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.383843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36caf94-ecf4-4b21-88d7-a4165a2ba106 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.386089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36caf94-ecf4-4b21-88d7-a4165a2ba106 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:44.387937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36caf94-ecf4-4b21-88d7-a4165a2ba106 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:44.390210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 181/358] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:44 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:44.391609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70421e54-594b-4aa6-abdb-952d236559b9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.391898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70421e54-594b-4aa6-abdb-952d236559b9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.392174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70421e54-594b-4aa6-abdb-952d236559b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.404274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da515026-2578-43a7-81f7-b73fcbd76823 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:44.410597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-70421e54-594b-4aa6-abdb-952d236559b9 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:44.411808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-70421e54-594b-4aa6-abdb-952d236559b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:44.446698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:44.455106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb77e3b-72b5-479a-aa07-9faee596c5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:44.455684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb77e3b-72b5-479a-aa07-9faee596c5aa None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.463223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb77e3b-72b5-479a-aa07-9faee596c5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.463223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb77e3b-72b5-479a-aa07-9faee596c5aa None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.463223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb77e3b-72b5-479a-aa07-9faee596c5aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:44.463223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb77e3b-72b5-479a-aa07-9faee596c5aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:44.463223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb77e3b-72b5-479a-aa07-9faee596c5aa None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:44.471892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 178/359] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:44.473639 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12253e-66d0-484e-8cab-e53c8701d475 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.473931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12253e-66d0-484e-8cab-e53c8701d475 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.474210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12253e-66d0-484e-8cab-e53c8701d475 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.475848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 179/360] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:44.477037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12253e-66d0-484e-8cab-e53c8701d475 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:44.511409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 182/361] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:44.517181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70421e54-594b-4aa6-abdb-952d236559b9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:44.595848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:44.616502 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12253e-66d0-484e-8cab-e53c8701d475 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:44.657276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059f74d-d29d-4aaf-b078-07b17ed2e04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:44.658418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059f74d-d29d-4aaf-b078-07b17ed2e04d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.658713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059f74d-d29d-4aaf-b078-07b17ed2e04d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.658982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059f74d-d29d-4aaf-b078-07b17ed2e04d None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.659810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70421e54-594b-4aa6-abdb-952d236559b9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:44.660221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059f74d-d29d-4aaf-b078-07b17ed2e04d None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:44.661970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059f74d-d29d-4aaf-b078-07b17ed2e04d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:44.665393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 180/362] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:44 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:44.665946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059f74d-d29d-4aaf-b078-07b17ed2e04d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:44.704483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85095e66-d115-42a5-ba77-9af70e64ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:44.705376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85095e66-d115-42a5-ba77-9af70e64ca45 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.705654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85095e66-d115-42a5-ba77-9af70e64ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.705920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85095e66-d115-42a5-ba77-9af70e64ca45 None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.707093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85095e66-d115-42a5-ba77-9af70e64ca45 None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:44.708524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85095e66-d115-42a5-ba77-9af70e64ca45 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:44.712607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 183/363] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:44 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:44.712607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7e4d8-8aab-4c73-b755-d509c1ae5741 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.712607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7e4d8-8aab-4c73-b755-d509c1ae5741 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.712607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7e4d8-8aab-4c73-b755-d509c1ae5741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.731015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f5d7e4d8-8aab-4c73-b755-d509c1ae5741 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:44.732191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f5d7e4d8-8aab-4c73-b755-d509c1ae5741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:44.758690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:44.811329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059f74d-d29d-4aaf-b078-07b17ed2e04d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:44.823045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 184/364] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:44.824104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7e4d8-8aab-4c73-b755-d509c1ae5741 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:44.881611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed32610-0a15-43eb-a161-df1a416e62e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:44.881611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed32610-0a15-43eb-a161-df1a416e62e3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:44.881611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed32610-0a15-43eb-a161-df1a416e62e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:44.881611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed32610-0a15-43eb-a161-df1a416e62e3 None admin] PATH_INFO: `/v3/users/d765d8f918cd4155bb0209ba9e4ee9ad` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:44.882302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed32610-0a15-43eb-a161-df1a416e62e3 None admin] RBAC: Authorizing `identity:get_user(user_id=d765d8f918cd4155bb0209ba9e4ee9ad)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:44.882302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed32610-0a15-43eb-a161-df1a416e62e3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:44.882451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 181/365] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:23:44 2022] GET /identity/v3/users/d765d8f918cd4155bb0209ba9e4ee9ad => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:44.883655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed32610-0a15-43eb-a161-df1a416e62e3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:44.897611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:44.960675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7e4d8-8aab-4c73-b755-d509c1ae5741 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:45.001266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37ecec-e23f-4125-a39d-8714da5c24eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:45.002087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ecec-e23f-4125-a39d-8714da5c24eb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:45.002390 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ecec-e23f-4125-a39d-8714da5c24eb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:45.002631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ecec-e23f-4125-a39d-8714da5c24eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:45.008163 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed32610-0a15-43eb-a161-df1a416e62e3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:45.018816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37ecec-e23f-4125-a39d-8714da5c24eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3c37ecec-e23f-4125-a39d-8714da5c24eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:45.027600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:45.028506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:45.029489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 185/366] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:45.030784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37ecec-e23f-4125-a39d-8714da5c24eb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:45.049539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:45.056210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67b4e8-f838-41e3-ab6f-6cab6a5841d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:45.056869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67b4e8-f838-41e3-ab6f-6cab6a5841d1 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:45.065120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67b4e8-f838-41e3-ab6f-6cab6a5841d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:45.065120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67b4e8-f838-41e3-ab6f-6cab6a5841d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:45.065120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67b4e8-f838-41e3-ab6f-6cab6a5841d1 None admin] RBAC: Adding query filter params (group.id=8aef3e683eb24cba992d6f7b8d076340, role.id=df5932e731294f7498a6983a730628bd, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:45.065120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67b4e8-f838-41e3-ab6f-6cab6a5841d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:45.065120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67b4e8-f838-41e3-ab6f-6cab6a5841d1 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:45.066018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 182/367] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jul 14 11:23:44 2022] GET /identity/v3/role_assignments?group.id=8aef3e683eb24cba992d6f7b8d076340&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=df5932e731294f7498a6983a730628bd => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:45.066832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67b4e8-f838-41e3-ab6f-6cab6a5841d1 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:45.177461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37ecec-e23f-4125-a39d-8714da5c24eb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:45.198343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:45.206003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67b4e8-f838-41e3-ab6f-6cab6a5841d1 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:45.236982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23894638-229f-4858-8034-16ba7e52fe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:45.238261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23894638-229f-4858-8034-16ba7e52fe2c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:45.239180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23894638-229f-4858-8034-16ba7e52fe2c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:45.239822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23894638-229f-4858-8034-16ba7e52fe2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:45.240970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23894638-229f-4858-8034-16ba7e52fe2c None admin] RBAC: Adding query filter params (group.id=8aef3e683eb24cba992d6f7b8d076340, role.id=41fd0da9b16d41c08992073473d87488, scope.project.id=d76703b19c6c4521a9480e46863c1979) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:45.241709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23894638-229f-4858-8034-16ba7e52fe2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:45.244069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23894638-229f-4858-8034-16ba7e52fe2c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:45.252282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889789f7-2859-47c0-a942-3ec721ab7cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:45.252540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 186/368] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jul 14 11:23:45 2022] GET /identity/v3/role_assignments?group.id=8aef3e683eb24cba992d6f7b8d076340&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=41fd0da9b16d41c08992073473d87488 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:45.253196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889789f7-2859-47c0-a942-3ec721ab7cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:45.253603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889789f7-2859-47c0-a942-3ec721ab7cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:45.253875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23894638-229f-4858-8034-16ba7e52fe2c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:45.254058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889789f7-2859-47c0-a942-3ec721ab7cf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:45.276650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889789f7-2859-47c0-a942-3ec721ab7cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-889789f7-2859-47c0-a942-3ec721ab7cf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:45.284905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:45.285925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:45.286717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 183/369] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:23:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:45.287382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889789f7-2859-47c0-a942-3ec721ab7cf1 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:45.393591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:45.455797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23894638-229f-4858-8034-16ba7e52fe2c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:45.482968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889789f7-2859-47c0-a942-3ec721ab7cf1 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:45.521280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b2dd7-5590-4874-87e0-29f2e2dbbd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:45.522560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2dd7-5590-4874-87e0-29f2e2dbbd62 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:45.523040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2dd7-5590-4874-87e0-29f2e2dbbd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:45.523826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2dd7-5590-4874-87e0-29f2e2dbbd62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:45.525112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b2dd7-5590-4874-87e0-29f2e2dbbd62 None admin] RBAC: Adding query filter params (role.id=4dc8f8c1b32c4c57805640c209616fde, user.id=d765d8f918cd4155bb0209ba9e4ee9ad, scope.system=all) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:45.525779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b2dd7-5590-4874-87e0-29f2e2dbbd62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:45.528022 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b2dd7-5590-4874-87e0-29f2e2dbbd62 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:45.541085 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cc6c1-e9e6-436b-b3ba-0ec3a08b0b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:45.541787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cc6c1-e9e6-436b-b3ba-0ec3a08b0b93 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:45.542066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cc6c1-e9e6-436b-b3ba-0ec3a08b0b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:45.542340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cc6c1-e9e6-436b-b3ba-0ec3a08b0b93 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:45.543095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168cc6c1-e9e6-436b-b3ba-0ec3a08b0b93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:45.543400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168cc6c1-e9e6-436b-b3ba-0ec3a08b0b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:45.546198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168cc6c1-e9e6-436b-b3ba-0ec3a08b0b93 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:45.549396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 187/370] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jul 14 11:23:45 2022] GET /identity/v3/role_assignments?user.id=d765d8f918cd4155bb0209ba9e4ee9ad&scope.system=all&role.id=4dc8f8c1b32c4c57805640c209616fde => generated 510 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:45.550452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b2dd7-5590-4874-87e0-29f2e2dbbd62 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:45.569461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 184/371] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:45.578414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cc6c1-e9e6-436b-b3ba-0ec3a08b0b93 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:45.627077 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:45.720453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b2dd7-5590-4874-87e0-29f2e2dbbd62 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:45.738665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cc6c1-e9e6-436b-b3ba-0ec3a08b0b93 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:45.760160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:45.776908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ddcc3-b37d-470d-aaf8-83c49dec5d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:45.777650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ddcc3-b37d-470d-aaf8-83c49dec5d09 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:45.777930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ddcc3-b37d-470d-aaf8-83c49dec5d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:45.778209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ddcc3-b37d-470d-aaf8-83c49dec5d09 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:45.778990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ddcc3-b37d-470d-aaf8-83c49dec5d09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:45.779298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ddcc3-b37d-470d-aaf8-83c49dec5d09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:45.780710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ddcc3-b37d-470d-aaf8-83c49dec5d09 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:45.794786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158b0ae-f916-490d-a9a3-5c9a62928eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:45.795450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158b0ae-f916-490d-a9a3-5c9a62928eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:45.798658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158b0ae-f916-490d-a9a3-5c9a62928eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:45.798658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158b0ae-f916-490d-a9a3-5c9a62928eb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:45.802573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 188/372] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:45.810387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ddcc3-b37d-470d-aaf8-83c49dec5d09 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:45.932591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158b0ae-f916-490d-a9a3-5c9a62928eb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:45.935492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158b0ae-f916-490d-a9a3-5c9a62928eb0 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:46.012399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ddcc3-b37d-470d-aaf8-83c49dec5d09 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:46.038226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:46.055026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a14b6-ae5e-4506-89d0-1a1964ae8df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:46.055693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a14b6-ae5e-4506-89d0-1a1964ae8df0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:46.055984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a14b6-ae5e-4506-89d0-1a1964ae8df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:46.056251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a14b6-ae5e-4506-89d0-1a1964ae8df0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:46.090463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 185/373] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:46.095869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158b0ae-f916-490d-a9a3-5c9a62928eb0 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:46.116660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:46.137099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a14b6-ae5e-4506-89d0-1a1964ae8df0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:46.138663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a14b6-ae5e-4506-89d0-1a1964ae8df0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:46.238473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158b0ae-f916-490d-a9a3-5c9a62928eb0 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:46.244946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:46.251138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 189/374] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:46.279550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a14b6-ae5e-4506-89d0-1a1964ae8df0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:46.319508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6e9b8-47c3-4428-ae09-b5be79774e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:46.320377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6e9b8-47c3-4428-ae09-b5be79774e15 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:46.320659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6e9b8-47c3-4428-ae09-b5be79774e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:46.320936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6e9b8-47c3-4428-ae09-b5be79774e15 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:46.323532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6e9b8-47c3-4428-ae09-b5be79774e15 None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:46.325002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6e9b8-47c3-4428-ae09-b5be79774e15 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:46.331855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 186/375] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:46 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:46.337615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6e9b8-47c3-4428-ae09-b5be79774e15 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:46.453780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a14b6-ae5e-4506-89d0-1a1964ae8df0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:46.500466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e2412-90bd-4005-912a-9c5fd996b74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:46.501374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e2412-90bd-4005-912a-9c5fd996b74d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:46.501655 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e2412-90bd-4005-912a-9c5fd996b74d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:46.501930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e2412-90bd-4005-912a-9c5fd996b74d None admin] PATH_INFO: `/v3/projects/da372d98ddc34559ac820df9de6d2798` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:46.503136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e2412-90bd-4005-912a-9c5fd996b74d None admin] RBAC: Authorizing `identity:get_project(project_id=da372d98ddc34559ac820df9de6d2798)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:46.507572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e2412-90bd-4005-912a-9c5fd996b74d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:46.507572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 190/376] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:46 2022] GET /identity/v3/projects/da372d98ddc34559ac820df9de6d2798 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:46.513195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e2412-90bd-4005-912a-9c5fd996b74d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:46.524995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:46.600974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:46.603384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6e9b8-47c3-4428-ae09-b5be79774e15 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:46.632319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e2412-90bd-4005-912a-9c5fd996b74d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:46.652352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88438c5d-3669-4c0b-8293-516aa4fcd8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:46.653316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88438c5d-3669-4c0b-8293-516aa4fcd8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:46.653598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88438c5d-3669-4c0b-8293-516aa4fcd8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:46.653873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88438c5d-3669-4c0b-8293-516aa4fcd8b7 None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:46.655129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88438c5d-3669-4c0b-8293-516aa4fcd8b7 None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:46.656593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88438c5d-3669-4c0b-8293-516aa4fcd8b7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:46.658825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 187/377] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:46 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:46.660144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122b322-72c4-4fb3-90c3-03f1b46d4342 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:46.660423 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122b322-72c4-4fb3-90c3-03f1b46d4342 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:46.660692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122b322-72c4-4fb3-90c3-03f1b46d4342 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:46.663948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 188/378] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:46.664727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122b322-72c4-4fb3-90c3-03f1b46d4342 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:46.685807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24134f-5f8c-414c-95e5-ade3b9694c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:46.686674 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24134f-5f8c-414c-95e5-ade3b9694c7b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:46.686949 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24134f-5f8c-414c-95e5-ade3b9694c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:46.687217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24134f-5f8c-414c-95e5-ade3b9694c7b None admin] PATH_INFO: `/v3/groups/c601d44c103c47809ee535295c1a14b6` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:46.702277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24134f-5f8c-414c-95e5-ade3b9694c7b None admin] RBAC: Authorizing `identity:get_group(group_id=c601d44c103c47809ee535295c1a14b6)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:46.704666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24134f-5f8c-414c-95e5-ade3b9694c7b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:46.707701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 191/379] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:46 2022] GET /identity/v3/groups/c601d44c103c47809ee535295c1a14b6 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:46.710666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a267c-4096-4712-b55f-35208877d8ef None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:46.711232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a267c-4096-4712-b55f-35208877d8ef None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:46.711675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a267c-4096-4712-b55f-35208877d8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:46.737311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-af2a267c-4096-4712-b55f-35208877d8ef None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:46.739856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-af2a267c-4096-4712-b55f-35208877d8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:46.814772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122b322-72c4-4fb3-90c3-03f1b46d4342 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:46.839973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 192/380] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:46.841463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a267c-4096-4712-b55f-35208877d8ef None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:46.857769 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf52048-da8d-4872-8c84-f4bf1d07b3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:46.858575 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf52048-da8d-4872-8c84-f4bf1d07b3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:46.858808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf52048-da8d-4872-8c84-f4bf1d07b3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:46.859083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf52048-da8d-4872-8c84-f4bf1d07b3b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:46.860323 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf52048-da8d-4872-8c84-f4bf1d07b3b0 None admin] RBAC: Adding query filter params (group.id=8aef3e683eb24cba992d6f7b8d076340, role.id=41fd0da9b16d41c08992073473d87488, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:46.860636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf52048-da8d-4872-8c84-f4bf1d07b3b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:46.862209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf52048-da8d-4872-8c84-f4bf1d07b3b0 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:46.868597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 189/381] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jul 14 11:23:46 2022] GET /identity/v3/role_assignments?group.id=8aef3e683eb24cba992d6f7b8d076340&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=41fd0da9b16d41c08992073473d87488 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:46.878231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf52048-da8d-4872-8c84-f4bf1d07b3b0 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:46.902445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:46.928980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:46.983305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a267c-4096-4712-b55f-35208877d8ef None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:47.012810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf52048-da8d-4872-8c84-f4bf1d07b3b0 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:47.030861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee148f-5f68-46eb-8602-ae51b414e1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:47.031691 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee148f-5f68-46eb-8602-ae51b414e1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:47.032068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee148f-5f68-46eb-8602-ae51b414e1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:47.032462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee148f-5f68-46eb-8602-ae51b414e1a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:47.033591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee148f-5f68-46eb-8602-ae51b414e1a6 None admin] RBAC: Adding query filter params (group.id=c601d44c103c47809ee535295c1a14b6, role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=da372d98ddc34559ac820df9de6d2798) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:47.033839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee148f-5f68-46eb-8602-ae51b414e1a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:47.035367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee148f-5f68-46eb-8602-ae51b414e1a6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:47.048875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 193/382] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jul 14 11:23:46 2022] GET /identity/v3/role_assignments?group.id=c601d44c103c47809ee535295c1a14b6&scope.project.id=da372d98ddc34559ac820df9de6d2798&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:47.068287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815e9cb-cfa4-4791-9a45-e72397f3bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:47.069932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815e9cb-cfa4-4791-9a45-e72397f3bce7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:47.070284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815e9cb-cfa4-4791-9a45-e72397f3bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:47.070647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815e9cb-cfa4-4791-9a45-e72397f3bce7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:47.083634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815e9cb-cfa4-4791-9a45-e72397f3bce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-d815e9cb-cfa4-4791-9a45-e72397f3bce7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:47.093267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:47.094235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:47.095036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 190/383] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:47.101077 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee148f-5f68-46eb-8602-ae51b414e1a6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:47.234806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee148f-5f68-46eb-8602-ae51b414e1a6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:47.269553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:47.283169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a301d37-eae9-4e33-9024-853978e78e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:47.284975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a301d37-eae9-4e33-9024-853978e78e89 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:47.284975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a301d37-eae9-4e33-9024-853978e78e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:47.284975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a301d37-eae9-4e33-9024-853978e78e89 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:47.285401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a301d37-eae9-4e33-9024-853978e78e89 None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:47.285612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a301d37-eae9-4e33-9024-853978e78e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:47.287062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a301d37-eae9-4e33-9024-853978e78e89 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:47.297286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 194/384] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:47.302677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815e9cb-cfa4-4791-9a45-e72397f3bce7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:47.401888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:47.439014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815e9cb-cfa4-4791-9a45-e72397f3bce7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:47.520184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f3336-cf18-4fc4-b5c5-7f33cc3f4cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:47.521052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f3336-cf18-4fc4-b5c5-7f33cc3f4cfb None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:47.521362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f3336-cf18-4fc4-b5c5-7f33cc3f4cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:47.521711 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f3336-cf18-4fc4-b5c5-7f33cc3f4cfb None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:47.522949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f3336-cf18-4fc4-b5c5-7f33cc3f4cfb None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:47.524406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f3336-cf18-4fc4-b5c5-7f33cc3f4cfb None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:47.526107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 191/385] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:47 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:47.534283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a301d37-eae9-4e33-9024-853978e78e89 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:47.684861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a301d37-eae9-4e33-9024-853978e78e89 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:47.731768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f79eae-8ab3-495d-ab28-7916783338b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:47.732626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f79eae-8ab3-495d-ab28-7916783338b0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:47.732901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f79eae-8ab3-495d-ab28-7916783338b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:47.733181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f79eae-8ab3-495d-ab28-7916783338b0 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:47.734365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f79eae-8ab3-495d-ab28-7916783338b0 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:47.735803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f79eae-8ab3-495d-ab28-7916783338b0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:47.737821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 195/386] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:47 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:47.744032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f3336-cf18-4fc4-b5c5-7f33cc3f4cfb None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:47.907126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f3336-cf18-4fc4-b5c5-7f33cc3f4cfb None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:47.948938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf6c1-7384-4963-bcaa-d9abc05166f3 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:47.949287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf6c1-7384-4963-bcaa-d9abc05166f3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:47.949523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf6c1-7384-4963-bcaa-d9abc05166f3 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:47.951141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 196/387] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:23:47.956560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3818af-9489-4b2c-9472-4428aea6c2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:47.957275 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ac860-bc9b-420a-99ac-b6338d2fa525 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:47.957561 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ac860-bc9b-420a-99ac-b6338d2fa525 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:47.957834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ac860-bc9b-420a-99ac-b6338d2fa525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:47.958330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3818af-9489-4b2c-9472-4428aea6c2a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:47.958819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3818af-9489-4b2c-9472-4428aea6c2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:47.959315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3818af-9489-4b2c-9472-4428aea6c2a0 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979/groups/8aef3e683eb24cba992d6f7b8d076340/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:47.965090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3818af-9489-4b2c-9472-4428aea6c2a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d76703b19c6c4521a9480e46863c1979, group_id=8aef3e683eb24cba992d6f7b8d076340, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:47.966812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3818af-9489-4b2c-9472-4428aea6c2a0 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:47.978076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d51ac860-bc9b-420a-99ac-b6338d2fa525 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:47.979637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d51ac860-bc9b-420a-99ac-b6338d2fa525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:48.033025 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:48.050778 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 197/388] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:48.054377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f708d-a767-4cb5-9443-1e0c6423c79c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:48.054377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f708d-a767-4cb5-9443-1e0c6423c79c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:48.054377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f708d-a767-4cb5-9443-1e0c6423c79c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:48.057946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 198/389] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:48.061187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 192/390] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jul 14 11:23:47 2022] PUT /identity/v3/projects/d76703b19c6c4521a9480e46863c1979/groups/8aef3e683eb24cba992d6f7b8d076340/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:48.074377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1741439-71a9-41e3-acd9-01ad24c65e4f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:48.074377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1741439-71a9-41e3-acd9-01ad24c65e4f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:48.074377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1741439-71a9-41e3-acd9-01ad24c65e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:48.091999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e1741439-71a9-41e3-acd9-01ad24c65e4f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:48.093221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e1741439-71a9-41e3-acd9-01ad24c65e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:48.102433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3818af-9489-4b2c-9472-4428aea6c2a0 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:48.120774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:48.176321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 199/391] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:48.199469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1741439-71a9-41e3-acd9-01ad24c65e4f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:48.310064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1741439-71a9-41e3-acd9-01ad24c65e4f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:48.326070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:48.344648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3818af-9489-4b2c-9472-4428aea6c2a0 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:48.348139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2eb27-75de-48ad-a69a-475cedc65100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:48.349012 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2eb27-75de-48ad-a69a-475cedc65100 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:48.349340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2eb27-75de-48ad-a69a-475cedc65100 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:48.349716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2eb27-75de-48ad-a69a-475cedc65100 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:48.362780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2eb27-75de-48ad-a69a-475cedc65100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-6ad2eb27-75de-48ad-a69a-475cedc65100 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:48.372140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:48.373051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:48.374160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:48.379538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 200/392] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:48.380574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2eb27-75de-48ad-a69a-475cedc65100 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:48.389503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cc8fd-1e70-4d53-ac11-2f01394609ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:48.391131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cc8fd-1e70-4d53-ac11-2f01394609ed None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:48.391702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cc8fd-1e70-4d53-ac11-2f01394609ed None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:48.392170 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cc8fd-1e70-4d53-ac11-2f01394609ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:48.412004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905cc8fd-1e70-4d53-ac11-2f01394609ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-905cc8fd-1e70-4d53-ac11-2f01394609ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:48.423233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:48.424184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:48.424979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:48.426938 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 193/393] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:48.437435 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cc8fd-1e70-4d53-ac11-2f01394609ed None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:48.545904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2eb27-75de-48ad-a69a-475cedc65100 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:48.555410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:48.608034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d0130-56ef-49ed-a12b-cd3b63b1843d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:48.608615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d0130-56ef-49ed-a12b-cd3b63b1843d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:48.608892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d0130-56ef-49ed-a12b-cd3b63b1843d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:48.609186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d0130-56ef-49ed-a12b-cd3b63b1843d None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:48.609933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d0130-56ef-49ed-a12b-cd3b63b1843d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:48.610232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d0130-56ef-49ed-a12b-cd3b63b1843d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:48.612371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d0130-56ef-49ed-a12b-cd3b63b1843d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:48.617461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cc8fd-1e70-4d53-ac11-2f01394609ed None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:48.625050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 201/394] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:48.630511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d0130-56ef-49ed-a12b-cd3b63b1843d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:48.666960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:48.682909 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5cd2-5b7b-4bea-a589-2ed49e4aef58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:48.683886 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5cd2-5b7b-4bea-a589-2ed49e4aef58 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:48.684365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5cd2-5b7b-4bea-a589-2ed49e4aef58 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:48.684850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5cd2-5b7b-4bea-a589-2ed49e4aef58 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:48.685881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c5cd2-5b7b-4bea-a589-2ed49e4aef58 None admin] RBAC: Adding query filter params (name=member) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:48.692469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c5cd2-5b7b-4bea-a589-2ed49e4aef58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:48.692469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c5cd2-5b7b-4bea-a589-2ed49e4aef58 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:48.708856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 194/395] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:48.715979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5cd2-5b7b-4bea-a589-2ed49e4aef58 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:48.743602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d0130-56ef-49ed-a12b-cd3b63b1843d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:48.782214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc046f7-5d78-40dd-be50-02187fe7f8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:48.783090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc046f7-5d78-40dd-be50-02187fe7f8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:48.783367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc046f7-5d78-40dd-be50-02187fe7f8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:48.783636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc046f7-5d78-40dd-be50-02187fe7f8d7 None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:48.784886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc046f7-5d78-40dd-be50-02187fe7f8d7 None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:48.786379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc046f7-5d78-40dd-be50-02187fe7f8d7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:48.788525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 202/396] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:48 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:48.794137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc046f7-5d78-40dd-be50-02187fe7f8d7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:48.941670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc046f7-5d78-40dd-be50-02187fe7f8d7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:48.944726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:48.971182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5cd2-5b7b-4bea-a589-2ed49e4aef58 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:48.977986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed6d22-366b-4f0b-8247-ab933cf864cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:48.978846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed6d22-366b-4f0b-8247-ab933cf864cb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:48.979122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed6d22-366b-4f0b-8247-ab933cf864cb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:48.979391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed6d22-366b-4f0b-8247-ab933cf864cb None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:48.980597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed6d22-366b-4f0b-8247-ab933cf864cb None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:48.982069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed6d22-366b-4f0b-8247-ab933cf864cb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:48.983777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 203/397] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:48 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:48.989395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed6d22-366b-4f0b-8247-ab933cf864cb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:49.019712 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:49.048572 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9c80c-6b55-45fc-8d35-f045819a2a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:49.049459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9c80c-6b55-45fc-8d35-f045819a2a75 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:49.049737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9c80c-6b55-45fc-8d35-f045819a2a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:49.050014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9c80c-6b55-45fc-8d35-f045819a2a75 None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:49.051219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9c80c-6b55-45fc-8d35-f045819a2a75 None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:49.061475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9c80c-6b55-45fc-8d35-f045819a2a75 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:49.063649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 195/398] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:48 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:49.070596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9c80c-6b55-45fc-8d35-f045819a2a75 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:49.098239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed6d22-366b-4f0b-8247-ab933cf864cb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:49.149192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69072492-99de-44dd-9397-277493faacfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:49.150092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072492-99de-44dd-9397-277493faacfa None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:49.150368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072492-99de-44dd-9397-277493faacfa None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:49.150856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072492-99de-44dd-9397-277493faacfa None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979/groups/8aef3e683eb24cba992d6f7b8d076340/roles/41fd0da9b16d41c08992073473d87488` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:49.157105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69072492-99de-44dd-9397-277493faacfa None admin] RBAC: Authorizing `identity:create_grant(project_id=d76703b19c6c4521a9480e46863c1979, group_id=8aef3e683eb24cba992d6f7b8d076340, role_id=41fd0da9b16d41c08992073473d87488)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:49.167334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69072492-99de-44dd-9397-277493faacfa None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:49.239967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9c80c-6b55-45fc-8d35-f045819a2a75 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:49.277149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4b089-f66a-44b3-9b39-a3c5554ca9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:49.278399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4b089-f66a-44b3-9b39-a3c5554ca9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:49.278910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4b089-f66a-44b3-9b39-a3c5554ca9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:49.279393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4b089-f66a-44b3-9b39-a3c5554ca9f5 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:49.280810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4b089-f66a-44b3-9b39-a3c5554ca9f5 None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:49.282478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4b089-f66a-44b3-9b39-a3c5554ca9f5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:49.284754 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 196/399] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:49 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:49.292812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4b089-f66a-44b3-9b39-a3c5554ca9f5 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:49.361389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:49.389878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 204/400] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jul 14 11:23:48 2022] PUT /identity/v3/projects/d76703b19c6c4521a9480e46863c1979/groups/8aef3e683eb24cba992d6f7b8d076340/roles/41fd0da9b16d41c08992073473d87488 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:23:49.405720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4b089-f66a-44b3-9b39-a3c5554ca9f5 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:49.407899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8e9e7-4948-4a0b-afbd-6ff35776f0c5 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:49.408138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8e9e7-4948-4a0b-afbd-6ff35776f0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:49.408387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8e9e7-4948-4a0b-afbd-6ff35776f0c5 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:49.410211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 205/401] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:49.416564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e312074-c46a-4877-a213-4f9651d760f6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:49.416877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e312074-c46a-4877-a213-4f9651d760f6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:49.417115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e312074-c46a-4877-a213-4f9651d760f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:49.452650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1e312074-c46a-4877-a213-4f9651d760f6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:49.454629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d653f6-3878-448d-83d5-5782c863551f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:49.455797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d653f6-3878-448d-83d5-5782c863551f None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:49.456301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d653f6-3878-448d-83d5-5782c863551f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:49.456786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d653f6-3878-448d-83d5-5782c863551f None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/groups/8aef3e683eb24cba992d6f7b8d076340/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:49.459407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1e312074-c46a-4877-a213-4f9651d760f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:49.462101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d653f6-3878-448d-83d5-5782c863551f None admin] RBAC: Authorizing `identity:create_grant(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9, group_id=8aef3e683eb24cba992d6f7b8d076340, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:49.463677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d653f6-3878-448d-83d5-5782c863551f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:49.495469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:49.543144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 206/402] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:49.565042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e312074-c46a-4877-a213-4f9651d760f6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:49.647975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 197/403] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jul 14 11:23:49 2022] PUT /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/groups/8aef3e683eb24cba992d6f7b8d076340/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:23:49.674762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e312074-c46a-4877-a213-4f9651d760f6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:49.681625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61060131-1bca-4968-af6d-e51ee50bfe8f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:49.698261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61060131-1bca-4968-af6d-e51ee50bfe8f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:49.698261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61060131-1bca-4968-af6d-e51ee50bfe8f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:49.698261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 198/404] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:49.698261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9696ca-c32d-48a4-8580-e3366dee9084 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:49.698261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9696ca-c32d-48a4-8580-e3366dee9084 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:49.698261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9696ca-c32d-48a4-8580-e3366dee9084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:49.725435 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-de9696ca-c32d-48a4-8580-e3366dee9084 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:49.728660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-de9696ca-c32d-48a4-8580-e3366dee9084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:49.733054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7fc58-ac4b-4d1d-bc42-0a57b4b0ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:49.734169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7fc58-ac4b-4d1d-bc42-0a57b4b0ec04 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:49.734608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7fc58-ac4b-4d1d-bc42-0a57b4b0ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:49.735037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7fc58-ac4b-4d1d-bc42-0a57b4b0ec04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:49.762766 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7fc58-ac4b-4d1d-bc42-0a57b4b0ec04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9ba7fc58-ac4b-4d1d-bc42-0a57b4b0ec04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:49.777592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:49.778618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:49.779680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 207/405] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:49.784836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7fc58-ac4b-4d1d-bc42-0a57b4b0ec04 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:49.851428 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 199/406] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:49.862911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:49.874222 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9696ca-c32d-48a4-8580-e3366dee9084 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:49.951104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7fc58-ac4b-4d1d-bc42-0a57b4b0ec04 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:49.953648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:50.012034 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9696ca-c32d-48a4-8580-e3366dee9084 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:50.045989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392ae0c-165b-46be-a644-463b8144e0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:50.046756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392ae0c-165b-46be-a644-463b8144e0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:50.046873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392ae0c-165b-46be-a644-463b8144e0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:50.047144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392ae0c-165b-46be-a644-463b8144e0a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:50.047910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392ae0c-165b-46be-a644-463b8144e0a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:50.048278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392ae0c-165b-46be-a644-463b8144e0a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:50.053953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392ae0c-165b-46be-a644-463b8144e0a0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:50.060980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51c4e-4126-4ed0-a1bd-3e61e000fe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:50.061827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51c4e-4126-4ed0-a1bd-3e61e000fe56 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:50.062363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51c4e-4126-4ed0-a1bd-3e61e000fe56 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:50.062363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd51c4e-4126-4ed0-a1bd-3e61e000fe56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:50.066558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 208/407] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:50.074779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392ae0c-165b-46be-a644-463b8144e0a0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:50.091183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd51c4e-4126-4ed0-a1bd-3e61e000fe56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-0cd51c4e-4126-4ed0-a1bd-3e61e000fe56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:50.109284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:50.110286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 200/408] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:23:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:50.111066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51c4e-4126-4ed0-a1bd-3e61e000fe56 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:50.234945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:50.267257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392ae0c-165b-46be-a644-463b8144e0a0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:50.316157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd51c4e-4126-4ed0-a1bd-3e61e000fe56 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:50.331647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40303a5e-1e17-426c-b0f7-29fcc9ebedb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:50.332510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40303a5e-1e17-426c-b0f7-29fcc9ebedb0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:50.332788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40303a5e-1e17-426c-b0f7-29fcc9ebedb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:50.333060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40303a5e-1e17-426c-b0f7-29fcc9ebedb0 None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:50.341177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40303a5e-1e17-426c-b0f7-29fcc9ebedb0 None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:50.341177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40303a5e-1e17-426c-b0f7-29fcc9ebedb0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:50.341177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 209/409] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:50 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:50.343557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40303a5e-1e17-426c-b0f7-29fcc9ebedb0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:50.350301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:50.392736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf11b-19e4-4f55-8f34-d79cb2c63695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:50.393446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf11b-19e4-4f55-8f34-d79cb2c63695 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:50.393734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf11b-19e4-4f55-8f34-d79cb2c63695 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:50.394013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf11b-19e4-4f55-8f34-d79cb2c63695 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:50.397601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcf11b-19e4-4f55-8f34-d79cb2c63695 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:50.397915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcf11b-19e4-4f55-8f34-d79cb2c63695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:50.399334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcf11b-19e4-4f55-8f34-d79cb2c63695 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:50.422300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 201/410] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:50.428389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf11b-19e4-4f55-8f34-d79cb2c63695 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:50.534090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40303a5e-1e17-426c-b0f7-29fcc9ebedb0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:50.583046 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf11b-19e4-4f55-8f34-d79cb2c63695 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:50.586791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba284e0-e0d6-4ad3-814d-7c70e64fb07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:50.587650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba284e0-e0d6-4ad3-814d-7c70e64fb07a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:50.587988 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba284e0-e0d6-4ad3-814d-7c70e64fb07a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:50.588259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba284e0-e0d6-4ad3-814d-7c70e64fb07a None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:50.589539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba284e0-e0d6-4ad3-814d-7c70e64fb07a None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:50.590976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba284e0-e0d6-4ad3-814d-7c70e64fb07a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:50.593004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 210/411] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:50 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:50.594357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627cec2-2923-4153-9920-c20a68d572af None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:50.594835 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627cec2-2923-4153-9920-c20a68d572af None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:50.595112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627cec2-2923-4153-9920-c20a68d572af None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:50.596588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 211/412] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:50.600212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627cec2-2923-4153-9920-c20a68d572af None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:50.622504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503ba7f-81b4-4133-acbd-1d7d0d3a8360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:50.623368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503ba7f-81b4-4133-acbd-1d7d0d3a8360 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:50.623647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503ba7f-81b4-4133-acbd-1d7d0d3a8360 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:50.624201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:50.624577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503ba7f-81b4-4133-acbd-1d7d0d3a8360 None admin] PATH_INFO: `/v3/groups/c601d44c103c47809ee535295c1a14b6` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:50.625811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503ba7f-81b4-4133-acbd-1d7d0d3a8360 None admin] RBAC: Authorizing `identity:get_group(group_id=c601d44c103c47809ee535295c1a14b6)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:50.627269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503ba7f-81b4-4133-acbd-1d7d0d3a8360 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:50.629309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 202/413] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:50 2022] GET /identity/v3/groups/c601d44c103c47809ee535295c1a14b6 => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:50.630574 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549cd12-5742-42cd-8c18-6e5a96d48797 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:50.630902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549cd12-5742-42cd-8c18-6e5a96d48797 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:50.631174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549cd12-5742-42cd-8c18-6e5a96d48797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:50.648902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-7549cd12-5742-42cd-8c18-6e5a96d48797 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:50.650197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7549cd12-5742-42cd-8c18-6e5a96d48797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:50.696336 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:50.730114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627cec2-2923-4153-9920-c20a68d572af None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:50.730345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 203/414] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:50.737402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549cd12-5742-42cd-8c18-6e5a96d48797 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:50.785378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da046e55-a7b1-46c8-bafa-a3b3f692225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:50.786260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046e55-a7b1-46c8-bafa-a3b3f692225b None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:50.786537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046e55-a7b1-46c8-bafa-a3b3f692225b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:50.786807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046e55-a7b1-46c8-bafa-a3b3f692225b None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/groups/8aef3e683eb24cba992d6f7b8d076340/roles/41fd0da9b16d41c08992073473d87488` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:50.789471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da046e55-a7b1-46c8-bafa-a3b3f692225b None admin] RBAC: Authorizing `identity:create_grant(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9, group_id=8aef3e683eb24cba992d6f7b8d076340, role_id=41fd0da9b16d41c08992073473d87488)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:50.790642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da046e55-a7b1-46c8-bafa-a3b3f692225b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:50.865272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549cd12-5742-42cd-8c18-6e5a96d48797 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:50.886422 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 212/415] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jul 14 11:23:50 2022] PUT /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9/groups/8aef3e683eb24cba992d6f7b8d076340/roles/41fd0da9b16d41c08992073473d87488 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:50.888067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da046e55-a7b1-46c8-bafa-a3b3f692225b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:50.906353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea4350-b775-47b9-9c55-973ad1ac32f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:50.907682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea4350-b775-47b9-9c55-973ad1ac32f9 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:50.908245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea4350-b775-47b9-9c55-973ad1ac32f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:50.908825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea4350-b775-47b9-9c55-973ad1ac32f9 None admin] PATH_INFO: `/v3/projects/da372d98ddc34559ac820df9de6d2798` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:50.910422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea4350-b775-47b9-9c55-973ad1ac32f9 None admin] RBAC: Authorizing `identity:get_project(project_id=da372d98ddc34559ac820df9de6d2798)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:50.912702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea4350-b775-47b9-9c55-973ad1ac32f9 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:50.925224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 204/416] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:50 2022] GET /identity/v3/projects/da372d98ddc34559ac820df9de6d2798 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:50.926832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea4350-b775-47b9-9c55-973ad1ac32f9 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:50.994019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:51.006965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:51.019263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da046e55-a7b1-46c8-bafa-a3b3f692225b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:51.060856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f71465-dae4-4d9c-baad-1b5ad2598c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:51.061994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f71465-dae4-4d9c-baad-1b5ad2598c8c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:51.062532 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f71465-dae4-4d9c-baad-1b5ad2598c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:51.062961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f71465-dae4-4d9c-baad-1b5ad2598c8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:51.078295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f71465-dae4-4d9c-baad-1b5ad2598c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-06f71465-dae4-4d9c-baad-1b5ad2598c8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:51.089179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:51.090305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:51.091293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:51.093881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 213/417] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:51.099758 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f71465-dae4-4d9c-baad-1b5ad2598c8c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:51.119638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea4350-b775-47b9-9c55-973ad1ac32f9 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:51.159460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84cca4-d739-4933-97b3-8b8e5e23b68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:51.160647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84cca4-d739-4933-97b3-8b8e5e23b68c None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:51.161149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84cca4-d739-4933-97b3-8b8e5e23b68c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:51.161649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84cca4-d739-4933-97b3-8b8e5e23b68c None admin] PATH_INFO: `/v3/projects/da372d98ddc34559ac820df9de6d2798/groups/c601d44c103c47809ee535295c1a14b6/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:51.164412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84cca4-d739-4933-97b3-8b8e5e23b68c None admin] RBAC: Authorizing `identity:create_grant(project_id=da372d98ddc34559ac820df9de6d2798, group_id=c601d44c103c47809ee535295c1a14b6, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:51.165890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84cca4-d739-4933-97b3-8b8e5e23b68c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:51.236527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f71465-dae4-4d9c-baad-1b5ad2598c8c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:51.279876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45b0a8-96f7-4d76-b9c7-16b5fdbfa52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:51.280953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45b0a8-96f7-4d76-b9c7-16b5fdbfa52a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:51.281412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 205/418] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jul 14 11:23:50 2022] PUT /identity/v3/projects/da372d98ddc34559ac820df9de6d2798/groups/c601d44c103c47809ee535295c1a14b6/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:51.281818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45b0a8-96f7-4d76-b9c7-16b5fdbfa52a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:51.282316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45b0a8-96f7-4d76-b9c7-16b5fdbfa52a None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:51.283370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45b0a8-96f7-4d76-b9c7-16b5fdbfa52a None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:51.284224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45b0a8-96f7-4d76-b9c7-16b5fdbfa52a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:51.286345 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45b0a8-96f7-4d76-b9c7-16b5fdbfa52a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:51.298516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 214/419] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:51.306143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84cca4-d739-4933-97b3-8b8e5e23b68c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:51.445817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:51.452280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84cca4-d739-4933-97b3-8b8e5e23b68c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:51.496802 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0be11-bc27-485e-b5aa-938251834327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:51.498019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0be11-bc27-485e-b5aa-938251834327 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:51.498560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0be11-bc27-485e-b5aa-938251834327 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:51.501598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0be11-bc27-485e-b5aa-938251834327 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:51.591289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0be11-bc27-485e-b5aa-938251834327 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:51.593150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0be11-bc27-485e-b5aa-938251834327 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:51.705633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:51.710149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 206/420] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:51.715727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45b0a8-96f7-4d76-b9c7-16b5fdbfa52a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:51.720681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:51.809856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc319f-0502-4527-a4c3-0fb9d5dba9aa None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:51.810476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc319f-0502-4527-a4c3-0fb9d5dba9aa None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:51.810966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc319f-0502-4527-a4c3-0fb9d5dba9aa None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:51.812824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 207/421] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:23:51.818897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b42cc-7ef2-49a0-99c3-4cbde4155789 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:51.819463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b42cc-7ef2-49a0-99c3-4cbde4155789 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:51.819949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b42cc-7ef2-49a0-99c3-4cbde4155789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:51.831735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45b0a8-96f7-4d76-b9c7-16b5fdbfa52a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:51.838484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-6e9b42cc-7ef2-49a0-99c3-4cbde4155789 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:51.839967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6e9b42cc-7ef2-49a0-99c3-4cbde4155789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:51.871672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751be4ef-913b-44b8-a8d1-d111cc6b4611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:51.872574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751be4ef-913b-44b8-a8d1-d111cc6b4611 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:51.874024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751be4ef-913b-44b8-a8d1-d111cc6b4611 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:51.874024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751be4ef-913b-44b8-a8d1-d111cc6b4611 None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:51.875035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751be4ef-913b-44b8-a8d1-d111cc6b4611 None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:51.876684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751be4ef-913b-44b8-a8d1-d111cc6b4611 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:51.879094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 215/422] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:51 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:51.885198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751be4ef-913b-44b8-a8d1-d111cc6b4611 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:51.922845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 208/423] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:51.947546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b42cc-7ef2-49a0-99c3-4cbde4155789 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:52.001442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:52.065942 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751be4ef-913b-44b8-a8d1-d111cc6b4611 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:52.124532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b42cc-7ef2-49a0-99c3-4cbde4155789 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:52.147938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd04e4-08c6-4525-a547-5b021a46c305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:52.148931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd04e4-08c6-4525-a547-5b021a46c305 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:52.149267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd04e4-08c6-4525-a547-5b021a46c305 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:52.149632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd04e4-08c6-4525-a547-5b021a46c305 None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:52.150849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd04e4-08c6-4525-a547-5b021a46c305 None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:52.152434 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd04e4-08c6-4525-a547-5b021a46c305 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:52.154553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 216/424] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:51 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:52.155183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:52.157104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489e95-3307-4671-8367-3d346448beed None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:52.157669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489e95-3307-4671-8367-3d346448beed None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:52.158169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489e95-3307-4671-8367-3d346448beed None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:52.164107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef0715-d1d5-422c-a174-f04ef5f5e56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:52.164107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef0715-d1d5-422c-a174-f04ef5f5e56a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:52.164107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef0715-d1d5-422c-a174-f04ef5f5e56a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:52.164107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef0715-d1d5-422c-a174-f04ef5f5e56a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:52.164851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 217/425] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:52.165872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8489e95-3307-4671-8367-3d346448beed None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:52.176076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef0715-d1d5-422c-a174-f04ef5f5e56a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-25ef0715-d1d5-422c-a174-f04ef5f5e56a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:52.184982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:52.186171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:52.187193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:52.188750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 209/426] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:23:52.190206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f3e2f-fd51-457f-8613-0a81394866ec None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:52.190780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f3e2f-fd51-457f-8613-0a81394866ec None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:52.191065 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f3e2f-fd51-457f-8613-0a81394866ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:52.211495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-937f3e2f-fd51-457f-8613-0a81394866ec None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:52.213190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-937f3e2f-fd51-457f-8613-0a81394866ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:52.293190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 210/427] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:52.293190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f3e2f-fd51-457f-8613-0a81394866ec None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:52.305407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8489e95-3307-4671-8367-3d346448beed None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:52.342864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023f2ff-fc57-4e00-aeab-60947e15988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:52.343518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f2ff-fc57-4e00-aeab-60947e15988b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:52.343799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f2ff-fc57-4e00-aeab-60947e15988b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:52.344071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f2ff-fc57-4e00-aeab-60947e15988b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:52.344898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023f2ff-fc57-4e00-aeab-60947e15988b None admin] RBAC: Adding query filter params (group.id=8aef3e683eb24cba992d6f7b8d076340, role.id=df5932e731294f7498a6983a730628bd, scope.project.id=d76703b19c6c4521a9480e46863c1979) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:52.345230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023f2ff-fc57-4e00-aeab-60947e15988b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:52.346650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023f2ff-fc57-4e00-aeab-60947e15988b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:52.353804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 218/428] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jul 14 11:23:52 2022] GET /identity/v3/role_assignments?group.id=8aef3e683eb24cba992d6f7b8d076340&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=df5932e731294f7498a6983a730628bd => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:52.354709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023f2ff-fc57-4e00-aeab-60947e15988b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:52.357450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:52.428578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f3e2f-fd51-457f-8613-0a81394866ec None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:52.451983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:52.467554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023f2ff-fc57-4e00-aeab-60947e15988b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:52.468699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05901e-b3d2-4d95-bbfb-7a87338e1efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:52.469695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05901e-b3d2-4d95-bbfb-7a87338e1efd None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:52.470188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05901e-b3d2-4d95-bbfb-7a87338e1efd None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:52.470800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05901e-b3d2-4d95-bbfb-7a87338e1efd None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:52.471785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05901e-b3d2-4d95-bbfb-7a87338e1efd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:52.472307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05901e-b3d2-4d95-bbfb-7a87338e1efd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:52.474082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05901e-b3d2-4d95-bbfb-7a87338e1efd None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:52.484465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 211/429] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:52.494243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05901e-b3d2-4d95-bbfb-7a87338e1efd None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:52.513189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05cff2-da34-4e15-9cdb-013a406cb608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:52.513189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05cff2-da34-4e15-9cdb-013a406cb608 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:52.513189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05cff2-da34-4e15-9cdb-013a406cb608 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:52.513189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05cff2-da34-4e15-9cdb-013a406cb608 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:52.519661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05cff2-da34-4e15-9cdb-013a406cb608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-ff05cff2-da34-4e15-9cdb-013a406cb608 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:52.528179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:52.529288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 11:23:52.530123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:52.537659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 219/430] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:52.543540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05cff2-da34-4e15-9cdb-013a406cb608 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:52.622112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05901e-b3d2-4d95-bbfb-7a87338e1efd None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:52.648781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:52.663867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25a1db-9fd5-412c-a7ba-963b4145b6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:52.664685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a1db-9fd5-412c-a7ba-963b4145b6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:52.665343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a1db-9fd5-412c-a7ba-963b4145b6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:52.665629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a1db-9fd5-412c-a7ba-963b4145b6e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:52.674180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05cff2-da34-4e15-9cdb-013a406cb608 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:52.718549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2483b5-1174-4d46-95be-208ea0cd315e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:52.719641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2483b5-1174-4d46-95be-208ea0cd315e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:52.720086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2483b5-1174-4d46-95be-208ea0cd315e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:52.720536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2483b5-1174-4d46-95be-208ea0cd315e None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:52.721379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2483b5-1174-4d46-95be-208ea0cd315e None admin] RBAC: Adding query filter params (name=member) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:52.721950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2483b5-1174-4d46-95be-208ea0cd315e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:52.724089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2483b5-1174-4d46-95be-208ea0cd315e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:52.726601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:52.736850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 220/431] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:23:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:52.743803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2483b5-1174-4d46-95be-208ea0cd315e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:52.757322 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a1db-9fd5-412c-a7ba-963b4145b6e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:52.759253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a1db-9fd5-412c-a7ba-963b4145b6e3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:52.864347 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2483b5-1174-4d46-95be-208ea0cd315e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:52.886858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 212/432] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:52.892016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:52.892587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25a1db-9fd5-412c-a7ba-963b4145b6e3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:52.903478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b264f8b-76a8-4602-a342-e82d346b4fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:52.904766 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b264f8b-76a8-4602-a342-e82d346b4fbf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:52.905141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b264f8b-76a8-4602-a342-e82d346b4fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:52.905471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b264f8b-76a8-4602-a342-e82d346b4fbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:52.983963 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b264f8b-76a8-4602-a342-e82d346b4fbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:52.985557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b264f8b-76a8-4602-a342-e82d346b4fbf None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:52.986196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:53.006310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25a1db-9fd5-412c-a7ba-963b4145b6e3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:53.044226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca186b-cf26-415d-b67c-15e86b94ccce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:53.045626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca186b-cf26-415d-b67c-15e86b94ccce None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:53.046174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca186b-cf26-415d-b67c-15e86b94ccce None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:53.046654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca186b-cf26-415d-b67c-15e86b94ccce None admin] PATH_INFO: `/v3/projects/d76703b19c6c4521a9480e46863c1979` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:53.048314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca186b-cf26-415d-b67c-15e86b94ccce None admin] RBAC: Authorizing `identity:get_project(project_id=d76703b19c6c4521a9480e46863c1979)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:53.050091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca186b-cf26-415d-b67c-15e86b94ccce None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:53.052579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 213/433] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:52 2022] GET /identity/v3/projects/d76703b19c6c4521a9480e46863c1979 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:53.057983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca186b-cf26-415d-b67c-15e86b94ccce None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:53.093943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 221/434] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:53.102658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b264f8b-76a8-4602-a342-e82d346b4fbf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:53.131108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:53.173084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca186b-cf26-415d-b67c-15e86b94ccce None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:53.217049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d0258-c3e1-47fe-9be5-d967497216db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:53.218187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0258-c3e1-47fe-9be5-d967497216db None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:53.218641 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0258-c3e1-47fe-9be5-d967497216db None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:53.219058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0258-c3e1-47fe-9be5-d967497216db None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:53.220626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d0258-c3e1-47fe-9be5-d967497216db None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:53.222310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d0258-c3e1-47fe-9be5-d967497216db None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:53.225203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 214/435] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:53 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:53.229092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7637a0c-0d0f-4171-89e1-2d359630482f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:53.229631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7637a0c-0d0f-4171-89e1-2d359630482f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:53.230049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7637a0c-0d0f-4171-89e1-2d359630482f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:53.230556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b264f8b-76a8-4602-a342-e82d346b4fbf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:53.232003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 215/436] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:53.233093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7637a0c-0d0f-4171-89e1-2d359630482f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:53.260984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:53.280347 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47e2c7-e4a4-4ddc-b804-82c3f023406e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:53.281496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47e2c7-e4a4-4ddc-b804-82c3f023406e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:53.281885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47e2c7-e4a4-4ddc-b804-82c3f023406e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:53.282259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47e2c7-e4a4-4ddc-b804-82c3f023406e None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:53.283518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47e2c7-e4a4-4ddc-b804-82c3f023406e None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:53.284976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47e2c7-e4a4-4ddc-b804-82c3f023406e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:53.287366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 222/437] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:53 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:53.288741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585edf9-063a-4b7e-b2de-be476e6277c4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:53.289027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585edf9-063a-4b7e-b2de-be476e6277c4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:53.289328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585edf9-063a-4b7e-b2de-be476e6277c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:53.308375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e585edf9-063a-4b7e-b2de-be476e6277c4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:53.309540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e585edf9-063a-4b7e-b2de-be476e6277c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:53.367511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7637a0c-0d0f-4171-89e1-2d359630482f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:53.390098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 223/438] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:53.391301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585edf9-063a-4b7e-b2de-be476e6277c4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:53.411708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1b984-0a4e-4b12-9a3b-d9b4538afeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:53.413401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1b984-0a4e-4b12-9a3b-d9b4538afeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:53.413867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1b984-0a4e-4b12-9a3b-d9b4538afeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:53.414334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1b984-0a4e-4b12-9a3b-d9b4538afeb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:53.415892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1b984-0a4e-4b12-9a3b-d9b4538afeb8 None admin] RBAC: Adding query filter params (group.id=8aef3e683eb24cba992d6f7b8d076340, role.id=41fd0da9b16d41c08992073473d87488, scope.project.id=d76703b19c6c4521a9480e46863c1979) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:53.416749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1b984-0a4e-4b12-9a3b-d9b4538afeb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:53.418122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:53.418773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1b984-0a4e-4b12-9a3b-d9b4538afeb8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:53.425537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 216/439] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jul 14 11:23:53 2022] GET /identity/v3/role_assignments?group.id=8aef3e683eb24cba992d6f7b8d076340&scope.project.id=d76703b19c6c4521a9480e46863c1979&role.id=41fd0da9b16d41c08992073473d87488 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:53.426599 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1b984-0a4e-4b12-9a3b-d9b4538afeb8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:53.510896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585edf9-063a-4b7e-b2de-be476e6277c4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:53.536443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:53.557463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57177ef7-d04f-40bf-83a7-279d1de870ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:53.558506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57177ef7-d04f-40bf-83a7-279d1de870ea None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:53.558803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57177ef7-d04f-40bf-83a7-279d1de870ea None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:53.559150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57177ef7-d04f-40bf-83a7-279d1de870ea None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:53.559420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1b984-0a4e-4b12-9a3b-d9b4538afeb8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:53.560559 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57177ef7-d04f-40bf-83a7-279d1de870ea None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:53.563437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57177ef7-d04f-40bf-83a7-279d1de870ea None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:53.565673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 224/440] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:53 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:53.574956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57177ef7-d04f-40bf-83a7-279d1de870ea None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:53.603551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8350fe-93bc-4ac1-9efc-193584acc505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:53.604654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8350fe-93bc-4ac1-9efc-193584acc505 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:53.605075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8350fe-93bc-4ac1-9efc-193584acc505 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:53.605537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8350fe-93bc-4ac1-9efc-193584acc505 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:53.617400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8350fe-93bc-4ac1-9efc-193584acc505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-5d8350fe-93bc-4ac1-9efc-193584acc505 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:53.626269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:53.627386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:53.628396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 217/441] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:23:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:53.636828 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8350fe-93bc-4ac1-9efc-193584acc505 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:53.692092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:53.705626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57177ef7-d04f-40bf-83a7-279d1de870ea None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:53.751299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f5123-8ca2-4570-a389-96eeeb7f62e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:53.751969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f5123-8ca2-4570-a389-96eeeb7f62e8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:53.752279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f5123-8ca2-4570-a389-96eeeb7f62e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:53.752527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f5123-8ca2-4570-a389-96eeeb7f62e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:53.753450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f5123-8ca2-4570-a389-96eeeb7f62e8 None admin] RBAC: Adding query filter params (group.id=8aef3e683eb24cba992d6f7b8d076340, role.id=df5932e731294f7498a6983a730628bd, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:53.753762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f5123-8ca2-4570-a389-96eeeb7f62e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:53.755212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f5123-8ca2-4570-a389-96eeeb7f62e8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:53.761990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 225/442] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jul 14 11:23:53 2022] GET /identity/v3/role_assignments?group.id=8aef3e683eb24cba992d6f7b8d076340&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=df5932e731294f7498a6983a730628bd => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:53.770621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8350fe-93bc-4ac1-9efc-193584acc505 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:53.816785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea294ec4-5b4c-433a-9ca0-fe4724c5aa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:53.817408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294ec4-5b4c-433a-9ca0-fe4724c5aa74 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:53.817689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294ec4-5b4c-433a-9ca0-fe4724c5aa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:53.817959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea294ec4-5b4c-433a-9ca0-fe4724c5aa74 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:53.819257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea294ec4-5b4c-433a-9ca0-fe4724c5aa74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:53.819562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea294ec4-5b4c-433a-9ca0-fe4724c5aa74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:53.821849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea294ec4-5b4c-433a-9ca0-fe4724c5aa74 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:53.834622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 218/443] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:23:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:53.838901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f5123-8ca2-4570-a389-96eeeb7f62e8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:53.961398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f5123-8ca2-4570-a389-96eeeb7f62e8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:53.999249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825245a-6053-4c4f-80de-1ea505fdae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:54.000284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825245a-6053-4c4f-80de-1ea505fdae61 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:54.000568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825245a-6053-4c4f-80de-1ea505fdae61 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:54.000842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825245a-6053-4c4f-80de-1ea505fdae61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:54.028927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:54.073064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825245a-6053-4c4f-80de-1ea505fdae61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:54.074641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825245a-6053-4c4f-80de-1ea505fdae61 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:54.165669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc715d8f-6396-4db8-91f5-25c2de213b5f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:54.165947 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc715d8f-6396-4db8-91f5-25c2de213b5f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:54.166226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc715d8f-6396-4db8-91f5-25c2de213b5f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:54.167579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 219/444] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:54.175144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a3960-ed43-4e4c-a4bd-462c9145ec9b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:54.175144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a3960-ed43-4e4c-a4bd-462c9145ec9b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:54.175768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a3960-ed43-4e4c-a4bd-462c9145ec9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:54.183970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:54.185205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 226/445] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:23:54.190619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825245a-6053-4c4f-80de-1ea505fdae61 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:54.197282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-8f0a3960-ed43-4e4c-a4bd-462c9145ec9b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:54.198514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8f0a3960-ed43-4e4c-a4bd-462c9145ec9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:54.244760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:54.265754 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 220/446] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:54.288675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a3960-ed43-4e4c-a4bd-462c9145ec9b None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:54.288897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825245a-6053-4c4f-80de-1ea505fdae61 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:54.327823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f0a9d-a8bf-4538-b4ff-e0cc7d2719c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:54.328591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f0a9d-a8bf-4538-b4ff-e0cc7d2719c3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:54.328876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f0a9d-a8bf-4538-b4ff-e0cc7d2719c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:54.329147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f0a9d-a8bf-4538-b4ff-e0cc7d2719c3 None admin] PATH_INFO: `/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:54.330352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f0a9d-a8bf-4538-b4ff-e0cc7d2719c3 None admin] RBAC: Authorizing `identity:get_project(project_id=a7ebdba679af4b2ba6c4eb0f15fe25e9)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:54.331823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f0a9d-a8bf-4538-b4ff-e0cc7d2719c3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:54.333542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 227/447] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:54 2022] GET /identity/v3/projects/a7ebdba679af4b2ba6c4eb0f15fe25e9 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:54.338960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f0a9d-a8bf-4538-b4ff-e0cc7d2719c3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:54.390922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a3960-ed43-4e4c-a4bd-462c9145ec9b None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:54.432010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42edbd-6abf-41e3-bb3a-ab57650dfc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:54.433232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42edbd-6abf-41e3-bb3a-ab57650dfc8c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:54.433771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42edbd-6abf-41e3-bb3a-ab57650dfc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:54.434217 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42edbd-6abf-41e3-bb3a-ab57650dfc8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:54.436227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:54.438116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f0a9d-a8bf-4538-b4ff-e0cc7d2719c3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:54.445871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42edbd-6abf-41e3-bb3a-ab57650dfc8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-0c42edbd-6abf-41e3-bb3a-ab57650dfc8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:23:54.454129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:54.468983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 221/448] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:23:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:23:54.470856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42edbd-6abf-41e3-bb3a-ab57650dfc8c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:54.472227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:54.477032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791db630-c013-4a5d-bfd2-83c1a7b4c2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:54.477951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791db630-c013-4a5d-bfd2-83c1a7b4c2cc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:54.478229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791db630-c013-4a5d-bfd2-83c1a7b4c2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:54.478496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791db630-c013-4a5d-bfd2-83c1a7b4c2cc None admin] PATH_INFO: `/v3/groups/8aef3e683eb24cba992d6f7b8d076340` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:54.479691 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791db630-c013-4a5d-bfd2-83c1a7b4c2cc None admin] RBAC: Authorizing `identity:get_group(group_id=8aef3e683eb24cba992d6f7b8d076340)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:54.481196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791db630-c013-4a5d-bfd2-83c1a7b4c2cc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:54.482948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 228/449] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:54 2022] GET /identity/v3/groups/8aef3e683eb24cba992d6f7b8d076340 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:54.488674 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791db630-c013-4a5d-bfd2-83c1a7b4c2cc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:54.568100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42edbd-6abf-41e3-bb3a-ab57650dfc8c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:54.587978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791db630-c013-4a5d-bfd2-83c1a7b4c2cc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:54.602105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96b313-1259-41fc-86e1-f5bc69f68bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:54.602768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96b313-1259-41fc-86e1-f5bc69f68bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:54.603051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96b313-1259-41fc-86e1-f5bc69f68bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:54.603323 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96b313-1259-41fc-86e1-f5bc69f68bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:54.604071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96b313-1259-41fc-86e1-f5bc69f68bd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:54.604375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96b313-1259-41fc-86e1-f5bc69f68bd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:54.605793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96b313-1259-41fc-86e1-f5bc69f68bd8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:54.613710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 222/450] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:54.619838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96b313-1259-41fc-86e1-f5bc69f68bd8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:54.621716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b7062-d90d-4c2a-9b3e-dbeaead07bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:54.622538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b7062-d90d-4c2a-9b3e-dbeaead07bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:54.622829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b7062-d90d-4c2a-9b3e-dbeaead07bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:54.623891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b7062-d90d-4c2a-9b3e-dbeaead07bb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:54.624740 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b7062-d90d-4c2a-9b3e-dbeaead07bb5 None admin] RBAC: Adding query filter params (group.id=8aef3e683eb24cba992d6f7b8d076340, role.id=41fd0da9b16d41c08992073473d87488, scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:54.625058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b7062-d90d-4c2a-9b3e-dbeaead07bb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:54.626527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b7062-d90d-4c2a-9b3e-dbeaead07bb5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:54.632838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 229/451] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jul 14 11:23:54 2022] GET /identity/v3/role_assignments?group.id=8aef3e683eb24cba992d6f7b8d076340&scope.project.id=a7ebdba679af4b2ba6c4eb0f15fe25e9&role.id=41fd0da9b16d41c08992073473d87488 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:54.679792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:54.726014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96b313-1259-41fc-86e1-f5bc69f68bd8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:54.762228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb521d-052d-4cc0-8954-40f75afad92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:54.762882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb521d-052d-4cc0-8954-40f75afad92b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:54.763162 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb521d-052d-4cc0-8954-40f75afad92b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:54.763435 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb521d-052d-4cc0-8954-40f75afad92b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:54.838855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb521d-052d-4cc0-8954-40f75afad92b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:54.840135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb521d-052d-4cc0-8954-40f75afad92b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:54.897478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:54.942437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 223/452] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 11:23:54.948371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b7062-d90d-4c2a-9b3e-dbeaead07bb5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:55.007535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:55.051269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b7062-d90d-4c2a-9b3e-dbeaead07bb5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:55.083170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26d33c-6303-474f-b02e-ae7a18c6edc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:55.084529 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26d33c-6303-474f-b02e-ae7a18c6edc3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:55.084810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26d33c-6303-474f-b02e-ae7a18c6edc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:55.085084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26d33c-6303-474f-b02e-ae7a18c6edc3 None admin] PATH_INFO: `/v3/projects/da372d98ddc34559ac820df9de6d2798` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:55.086286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26d33c-6303-474f-b02e-ae7a18c6edc3 None admin] RBAC: Authorizing `identity:get_project(project_id=da372d98ddc34559ac820df9de6d2798)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:55.087704 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26d33c-6303-474f-b02e-ae7a18c6edc3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:55.089475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 230/453] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:23:54 2022] GET /identity/v3/projects/da372d98ddc34559ac820df9de6d2798 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:55.094957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb521d-052d-4cc0-8954-40f75afad92b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:55.189974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb521d-052d-4cc0-8954-40f75afad92b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:55.222933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2a840-d962-4a16-b0b1-5eae8e550e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:55.223802 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2a840-d962-4a16-b0b1-5eae8e550e75 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:55.224057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2a840-d962-4a16-b0b1-5eae8e550e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:55.224855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2a840-d962-4a16-b0b1-5eae8e550e75 None admin] PATH_INFO: `/v3/groups/c601d44c103c47809ee535295c1a14b6` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:55.226068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2a840-d962-4a16-b0b1-5eae8e550e75 None admin] RBAC: Authorizing `identity:get_group(group_id=c601d44c103c47809ee535295c1a14b6)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:55.227509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2a840-d962-4a16-b0b1-5eae8e550e75 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:55.229116 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 224/454] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jul 14 11:23:55 2022] GET /identity/v3/groups/c601d44c103c47809ee535295c1a14b6 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:55.234882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26d33c-6303-474f-b02e-ae7a18c6edc3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:55.343887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26d33c-6303-474f-b02e-ae7a18c6edc3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:55.365017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:55.375943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732773d-96bc-4765-a5ba-d5ee9aa697e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:55.376592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732773d-96bc-4765-a5ba-d5ee9aa697e6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:55.376872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732773d-96bc-4765-a5ba-d5ee9aa697e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:55.377147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732773d-96bc-4765-a5ba-d5ee9aa697e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:55.379081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732773d-96bc-4765-a5ba-d5ee9aa697e6 None admin] RBAC: Adding query filter params (group.id=c601d44c103c47809ee535295c1a14b6, role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=da372d98ddc34559ac820df9de6d2798) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:55.379081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732773d-96bc-4765-a5ba-d5ee9aa697e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:55.379777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732773d-96bc-4765-a5ba-d5ee9aa697e6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:55.385926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 231/455] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jul 14 11:23:55 2022] GET /identity/v3/role_assignments?group.id=c601d44c103c47809ee535295c1a14b6&scope.project.id=da372d98ddc34559ac820df9de6d2798&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:57.813264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbca46d-90e2-4c39-885a-9f95bc94d309 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:57.814214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbca46d-90e2-4c39-885a-9f95bc94d309 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:57.814694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbca46d-90e2-4c39-885a-9f95bc94d309 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:57.816530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 225/456] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:57.822611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ceda7-0855-4bea-ac93-0681e5e5f38b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:57.823452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ceda7-0855-4bea-ac93-0681e5e5f38b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:57.823934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ceda7-0855-4bea-ac93-0681e5e5f38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:57.843019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-db4ceda7-0855-4bea-ac93-0681e5e5f38b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:57.844531 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-db4ceda7-0855-4bea-ac93-0681e5e5f38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:57.920938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 232/457] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:57.947110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbca46d-90e2-4c39-885a-9f95bc94d309 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:57.979186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:58.032057 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a8f74-4904-4688-b95b-9fcf56e2dd7b None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.032377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a8f74-4904-4688-b95b-9fcf56e2dd7b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.032607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a8f74-4904-4688-b95b-9fcf56e2dd7b None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.034260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 233/458] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:58.044435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff7372-59e6-48b8-8ef5-a5f31b39041d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.044717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff7372-59e6-48b8-8ef5-a5f31b39041d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.044984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff7372-59e6-48b8-8ef5-a5f31b39041d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.069825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-32ff7372-59e6-48b8-8ef5-a5f31b39041d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:58.070947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-32ff7372-59e6-48b8-8ef5-a5f31b39041d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:58.113970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbca46d-90e2-4c39-885a-9f95bc94d309 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:58.185955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 234/459] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:23:58.186954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e99564-8581-4bb1-92c3-3c9335236f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:58.187777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e99564-8581-4bb1-92c3-3c9335236f83 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.188058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e99564-8581-4bb1-92c3-3c9335236f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.188331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e99564-8581-4bb1-92c3-3c9335236f83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.215963 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff7372-59e6-48b8-8ef5-a5f31b39041d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:58.218318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e99564-8581-4bb1-92c3-3c9335236f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:58.220335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e99564-8581-4bb1-92c3-3c9335236f83 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-13e99564-8581-4bb1-92c3-3c9335236f83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:23:58.247955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:58.248894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:58.249785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:23:58.249785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:23:58.249785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:23:58.249785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:58.249785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 226/460] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:58.257303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e99564-8581-4bb1-92c3-3c9335236f83 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:58.273817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:58.330822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff7372-59e6-48b8-8ef5-a5f31b39041d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:58.369469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62901c8c-8d3e-4166-bf6f-8779d5e2f041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:58.370521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62901c8c-8d3e-4166-bf6f-8779d5e2f041 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.370949 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62901c8c-8d3e-4166-bf6f-8779d5e2f041 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.371763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62901c8c-8d3e-4166-bf6f-8779d5e2f041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.383729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62901c8c-8d3e-4166-bf6f-8779d5e2f041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:58.385892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62901c8c-8d3e-4166-bf6f-8779d5e2f041 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-62901c8c-8d3e-4166-bf6f-8779d5e2f041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:23:58.406308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:58.407392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:58.408443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:23:58.408443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:23:58.408443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:23:58.408443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:58.410539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 235/461] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:23:58.417606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a23c79-7182-44e1-8588-cbc44281efee None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.417606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a23c79-7182-44e1-8588-cbc44281efee None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.417606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a23c79-7182-44e1-8588-cbc44281efee None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.420433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 236/462] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:58.421590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a23c79-7182-44e1-8588-cbc44281efee None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:58.451981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:58.486456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e99564-8581-4bb1-92c3-3c9335236f83 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:58.529355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7c757-ade6-4bec-8d64-691dc2e39bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:58.530058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7c757-ade6-4bec-8d64-691dc2e39bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.530337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7c757-ade6-4bec-8d64-691dc2e39bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.530645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7c757-ade6-4bec-8d64-691dc2e39bf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.531736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7c757-ade6-4bec-8d64-691dc2e39bf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:58.532049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7c757-ade6-4bec-8d64-691dc2e39bf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:58.534104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7c757-ade6-4bec-8d64-691dc2e39bf5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:58.556687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 227/463] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:58.556687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26951fbf-8bc7-496b-b2c2-fe7c6f28fc6e None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.556687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26951fbf-8bc7-496b-b2c2-fe7c6f28fc6e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.556687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26951fbf-8bc7-496b-b2c2-fe7c6f28fc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.574034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:58.577319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-26951fbf-8bc7-496b-b2c2-fe7c6f28fc6e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:58.578700 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-26951fbf-8bc7-496b-b2c2-fe7c6f28fc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:58.580671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a23c79-7182-44e1-8588-cbc44281efee None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:58.618547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e464da-eeaf-412d-a455-7313b5a086af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:58.619588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e464da-eeaf-412d-a455-7313b5a086af None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.620177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e464da-eeaf-412d-a455-7313b5a086af None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.620652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e464da-eeaf-412d-a455-7313b5a086af None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.621611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e464da-eeaf-412d-a455-7313b5a086af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:58.622113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e464da-eeaf-412d-a455-7313b5a086af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:58.623675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e464da-eeaf-412d-a455-7313b5a086af None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:58.640288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 237/464] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:58.640288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e464da-eeaf-412d-a455-7313b5a086af None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:58.651501 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 228/465] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:58.652945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98a1a-525c-4d6a-b3a9-c16197163f48 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.653271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98a1a-525c-4d6a-b3a9-c16197163f48 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.653553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98a1a-525c-4d6a-b3a9-c16197163f48 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.654965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 229/466] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:58.655573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c98a1a-525c-4d6a-b3a9-c16197163f48 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:58.729637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:58.763059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e464da-eeaf-412d-a455-7313b5a086af None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:58.766268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c98a1a-525c-4d6a-b3a9-c16197163f48 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:58.802030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1b711-3115-46ee-94d8-6704187dba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:58.803230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1b711-3115-46ee-94d8-6704187dba83 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.803507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1b711-3115-46ee-94d8-6704187dba83 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.803774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1b711-3115-46ee-94d8-6704187dba83 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.804480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1b711-3115-46ee-94d8-6704187dba83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:58.805879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e3e48-ab59-48c7-bd34-126328e54aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:58.806014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1b711-3115-46ee-94d8-6704187dba83 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:58.806575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e3e48-ab59-48c7-bd34-126328e54aef None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.806857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e3e48-ab59-48c7-bd34-126328e54aef None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.807131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e3e48-ab59-48c7-bd34-126328e54aef None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:58.807844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e3e48-ab59-48c7-bd34-126328e54aef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:58.809303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e3e48-ab59-48c7-bd34-126328e54aef None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:58.980656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 230/467] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:58 2022] POST /identity/v3/users => generated 248 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:58.982005 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa33878-0e2b-4d54-b546-bdce011fc1d0 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:58.984977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa33878-0e2b-4d54-b546-bdce011fc1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:58.984977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa33878-0e2b-4d54-b546-bdce011fc1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:59.001177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9fa33878-0e2b-4d54-b546-bdce011fc1d0 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:59.002685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9fa33878-0e2b-4d54-b546-bdce011fc1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:59.021843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 238/468] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:58 2022] POST /identity/v3/users => generated 246 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:59.023626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e3e48-ab59-48c7-bd34-126328e54aef None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:59.049214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:59.099638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 231/469] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:59.131278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa33878-0e2b-4d54-b546-bdce011fc1d0 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:59.185587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e3e48-ab59-48c7-bd34-126328e54aef None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:59.233192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2545b-ec61-4828-9487-dea8096c9386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:59.234345 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2545b-ec61-4828-9487-dea8096c9386 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:59.234623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2545b-ec61-4828-9487-dea8096c9386 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:59.234893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2545b-ec61-4828-9487-dea8096c9386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:59.237420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:59.250099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2545b-ec61-4828-9487-dea8096c9386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:59.251855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2545b-ec61-4828-9487-dea8096c9386 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-5bf2545b-ec61-4828-9487-dea8096c9386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:23:59.275246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:59.276161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:59.277027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:23:59.277027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:23:59.277027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:23:59.277027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:23:59.277027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 239/470] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:23:59.282166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2545b-ec61-4828-9487-dea8096c9386 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:59.296696 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa33878-0e2b-4d54-b546-bdce011fc1d0 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:59.343265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193e3ab-74a4-4473-9730-790158ab2684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:59.344728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193e3ab-74a4-4473-9730-790158ab2684 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:59.345013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193e3ab-74a4-4473-9730-790158ab2684 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:59.345337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193e3ab-74a4-4473-9730-790158ab2684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:59.365232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:59.366143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193e3ab-74a4-4473-9730-790158ab2684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:59.368420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193e3ab-74a4-4473-9730-790158ab2684 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-7193e3ab-74a4-4473-9730-790158ab2684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:23:59.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:23:59.390866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:23:59.392327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:23:59.392327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:23:59.392327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:23:59.392327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:23:59.392327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 232/471] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:23:59.396618 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193e3ab-74a4-4473-9730-790158ab2684 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:59.411001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2545b-ec61-4828-9487-dea8096c9386 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:59.456252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe668f0d-ed8b-4f62-a5eb-23a1afcfbbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:59.457374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe668f0d-ed8b-4f62-a5eb-23a1afcfbbcc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:59.457880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe668f0d-ed8b-4f62-a5eb-23a1afcfbbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:59.458463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe668f0d-ed8b-4f62-a5eb-23a1afcfbbcc None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:59.459504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe668f0d-ed8b-4f62-a5eb-23a1afcfbbcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:59.460018 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe668f0d-ed8b-4f62-a5eb-23a1afcfbbcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:59.461731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe668f0d-ed8b-4f62-a5eb-23a1afcfbbcc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:59.474613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 240/472] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:59.476337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7eda-d720-4fed-ba7b-7b38290fd343 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:59.476823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7eda-d720-4fed-ba7b-7b38290fd343 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:59.477356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7eda-d720-4fed-ba7b-7b38290fd343 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:59.479002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 241/473] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:23:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:23:59.479972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd7eda-d720-4fed-ba7b-7b38290fd343 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:59.511789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193e3ab-74a4-4473-9730-790158ab2684 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:59.514030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:59.548051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7116ae-4ef5-4049-9b68-8278e2cab6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:59.548453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7116ae-4ef5-4049-9b68-8278e2cab6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:59.548746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7116ae-4ef5-4049-9b68-8278e2cab6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:59.549026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7116ae-4ef5-4049-9b68-8278e2cab6c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:59.549847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7116ae-4ef5-4049-9b68-8278e2cab6c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:59.550100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7116ae-4ef5-4049-9b68-8278e2cab6c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:59.551569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7116ae-4ef5-4049-9b68-8278e2cab6c9 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:59.564125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 233/474] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:23:59.565526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966135b8-f72a-4d27-85d3-5bb4e66f93f6 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:59.565818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966135b8-f72a-4d27-85d3-5bb4e66f93f6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:59.566098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966135b8-f72a-4d27-85d3-5bb4e66f93f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:59.584910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-966135b8-f72a-4d27-85d3-5bb4e66f93f6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:59.586105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-966135b8-f72a-4d27-85d3-5bb4e66f93f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:59.600679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd7eda-d720-4fed-ba7b-7b38290fd343 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:59.629568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:59.642442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417f577-df1c-4497-8679-8e700dee181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:59.644461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417f577-df1c-4497-8679-8e700dee181b None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:59.644955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417f577-df1c-4497-8679-8e700dee181b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:59.645562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417f577-df1c-4497-8679-8e700dee181b None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:59.647066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417f577-df1c-4497-8679-8e700dee181b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:59.649286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417f577-df1c-4497-8679-8e700dee181b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:59.672874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 234/475] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:23:59.673541 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966135b8-f72a-4d27-85d3-5bb4e66f93f6 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:59.765116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 242/476] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:59 2022] POST /identity/v3/users => generated 248 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:59.766769 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417f577-df1c-4497-8679-8e700dee181b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:59.779191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:59.786377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966135b8-f72a-4d27-85d3-5bb4e66f93f6 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:59.867914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de97e9e-8dc2-49fc-aece-2d0e5e2114de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:59.870251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de97e9e-8dc2-49fc-aece-2d0e5e2114de None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:59.870795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de97e9e-8dc2-49fc-aece-2d0e5e2114de None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:59.871270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de97e9e-8dc2-49fc-aece-2d0e5e2114de None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:59.872467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de97e9e-8dc2-49fc-aece-2d0e5e2114de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:59.875165 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de97e9e-8dc2-49fc-aece-2d0e5e2114de None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:59.943236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417f577-df1c-4497-8679-8e700dee181b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:59.986319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68637266-c363-4043-8db1-79c1cc74df50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:59.987389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68637266-c363-4043-8db1-79c1cc74df50 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:59.987676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68637266-c363-4043-8db1-79c1cc74df50 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:59.987946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68637266-c363-4043-8db1-79c1cc74df50 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:59.988700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68637266-c363-4043-8db1-79c1cc74df50 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:59.990217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68637266-c363-4043-8db1-79c1cc74df50 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:00.094287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.sql.core [None req-68637266-c363-4043-8db1-79c1cc74df50 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 11:24:00.094287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 11:24:00.094287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [parameters: {'id': 'eedd01fc0fea4fcb99764cd06ec203e3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 11:24:00.094287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80433) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-68637266-c363-4043-8db1-79c1cc74df50 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application conn.query(q) Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result.read() Jul 14 11:24:00.111929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 11:24:00.112851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application next(self.gen) Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application yield resource Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application next(self.gen) Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise self.value Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application session.commit() Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 11:24:00.113837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.session.flush() Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application compat.raise_( Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 11:24:00.115222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 11:24:00.116097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application conn.query(q) Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result.read() Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 11:24:00.116906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [parameters: {'id': 'eedd01fc0fea4fcb99764cd06ec203e3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:00.119626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 11:24:00.121736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:00.123675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 243/477] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jul 14 11:23:59 2022] POST /identity/v3/roles => generated 144 bytes in 347 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:00.123675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68637266-c363-4043-8db1-79c1cc74df50 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:00.164826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 235/478] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:23:59 2022] POST /identity/v3/users => generated 249 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:00.172413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:00.247618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68637266-c363-4043-8db1-79c1cc74df50 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:00.303637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898fd2a-ec0d-4ec0-b2c9-67ccdd55977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:00.304488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898fd2a-ec0d-4ec0-b2c9-67ccdd55977d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:00.304773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898fd2a-ec0d-4ec0-b2c9-67ccdd55977d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:00.305043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898fd2a-ec0d-4ec0-b2c9-67ccdd55977d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:00.320637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898fd2a-ec0d-4ec0-b2c9-67ccdd55977d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9898fd2a-ec0d-4ec0-b2c9-67ccdd55977d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:00.329230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:00.330183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:00.330977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 244/479] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:24:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:00.337060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de97e9e-8dc2-49fc-aece-2d0e5e2114de None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:00.454173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de97e9e-8dc2-49fc-aece-2d0e5e2114de None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:00.496209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1a58a-5689-4e93-b278-d718ab009fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:00.496901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1a58a-5689-4e93-b278-d718ab009fbf None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:00.497223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1a58a-5689-4e93-b278-d718ab009fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:00.497476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1a58a-5689-4e93-b278-d718ab009fbf None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:00.498301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1a58a-5689-4e93-b278-d718ab009fbf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:00.498605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1a58a-5689-4e93-b278-d718ab009fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:00.500077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1a58a-5689-4e93-b278-d718ab009fbf None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:00.510362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 236/480] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:01.636379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84899533-973a-4d3e-97e6-5712247728bc None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:01.636899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84899533-973a-4d3e-97e6-5712247728bc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:01.636899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84899533-973a-4d3e-97e6-5712247728bc None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:01.638569 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 245/481] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:01.643850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd8351-e123-4403-ba1f-66d6b8d8b18e None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:01.644300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd8351-e123-4403-ba1f-66d6b8d8b18e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:01.644388 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd8351-e123-4403-ba1f-66d6b8d8b18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:01.666173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-6ffd8351-e123-4403-ba1f-66d6b8d8b18e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:01.667387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6ffd8351-e123-4403-ba1f-66d6b8d8b18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:01.676157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:01.729653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bd408-dbed-4d67-9f3b-48805a17709a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:01.729961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bd408-dbed-4d67-9f3b-48805a17709a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:01.730207 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bd408-dbed-4d67-9f3b-48805a17709a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:01.732330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 246/482] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:01.737845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09920ff-7529-4d27-bed5-67f460d46297 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:01.738121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09920ff-7529-4d27-bed5-67f460d46297 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:01.738652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09920ff-7529-4d27-bed5-67f460d46297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:01.754022 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 237/483] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:01.766264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b09920ff-7529-4d27-bed5-67f460d46297 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:01.767476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b09920ff-7529-4d27-bed5-67f460d46297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:01.777238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd8351-e123-4403-ba1f-66d6b8d8b18e None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:01.897985 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 247/484] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:24:01.926221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09920ff-7529-4d27-bed5-67f460d46297 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:01.971362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd8351-e123-4403-ba1f-66d6b8d8b18e None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:01.993518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:02.041345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a1e3-5766-4c39-b0a1-fa57538adf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:02.042270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a1e3-5766-4c39-b0a1-fa57538adf39 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.042448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a1e3-5766-4c39-b0a1-fa57538adf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.043156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a1e3-5766-4c39-b0a1-fa57538adf39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.054150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9a1e3-5766-4c39-b0a1-fa57538adf39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-c1f9a1e3-5766-4c39-b0a1-fa57538adf39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:02.062183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:02.063091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:02.064032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 238/485] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:02.074257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a1e3-5766-4c39-b0a1-fa57538adf39 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:02.089497 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:02.106207 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09920ff-7529-4d27-bed5-67f460d46297 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:02.155685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27639c9b-b9e2-46e5-bffc-af038d66e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:02.156827 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27639c9b-b9e2-46e5-bffc-af038d66e020 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.157486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27639c9b-b9e2-46e5-bffc-af038d66e020 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.159119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27639c9b-b9e2-46e5-bffc-af038d66e020 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.178384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27639c9b-b9e2-46e5-bffc-af038d66e020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-27639c9b-b9e2-46e5-bffc-af038d66e020 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:02.187492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:02.188442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:02.189271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:02.199013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 248/486] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:02.207737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27639c9b-b9e2-46e5-bffc-af038d66e020 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:02.270250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a1e3-5766-4c39-b0a1-fa57538adf39 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:02.314467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407feb9-a789-47c7-ac37-679449ecd7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:02.315685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407feb9-a789-47c7-ac37-679449ecd7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.316152 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407feb9-a789-47c7-ac37-679449ecd7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.316702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407feb9-a789-47c7-ac37-679449ecd7c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.319181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407feb9-a789-47c7-ac37-679449ecd7c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:02.319807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407feb9-a789-47c7-ac37-679449ecd7c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:02.320582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:02.322492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407feb9-a789-47c7-ac37-679449ecd7c5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:02.327602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27639c9b-b9e2-46e5-bffc-af038d66e020 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:02.334485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 239/487] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:02.340896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513a596-08b0-4ce2-bd65-3f99ce094b13 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.340896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513a596-08b0-4ce2-bd65-3f99ce094b13 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.340896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513a596-08b0-4ce2-bd65-3f99ce094b13 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.340896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 240/488] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:02.340896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513a596-08b0-4ce2-bd65-3f99ce094b13 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:02.385457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc7127-7b59-4f97-a900-3747fd7df47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:02.386423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc7127-7b59-4f97-a900-3747fd7df47c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.387005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc7127-7b59-4f97-a900-3747fd7df47c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.387435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc7127-7b59-4f97-a900-3747fd7df47c None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.388388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc7127-7b59-4f97-a900-3747fd7df47c None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:02.388854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc7127-7b59-4f97-a900-3747fd7df47c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:02.390472 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc7127-7b59-4f97-a900-3747fd7df47c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:02.400862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 249/489] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:02.402478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe1a7f-4e48-444e-b3e1-5d9715ccf298 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.402894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe1a7f-4e48-444e-b3e1-5d9715ccf298 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.403318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe1a7f-4e48-444e-b3e1-5d9715ccf298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.422091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5bbe1a7f-4e48-444e-b3e1-5d9715ccf298 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:02.423617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5bbe1a7f-4e48-444e-b3e1-5d9715ccf298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:02.429595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:02.472657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513a596-08b0-4ce2-bd65-3f99ce094b13 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:02.496098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 250/490] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:02.497441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe1a7f-4e48-444e-b3e1-5d9715ccf298 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:02.528450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92536631-57e7-4446-9f52-727013e1d6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:02.529381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92536631-57e7-4446-9f52-727013e1d6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.529666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92536631-57e7-4446-9f52-727013e1d6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.529944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92536631-57e7-4446-9f52-727013e1d6d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.548789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92536631-57e7-4446-9f52-727013e1d6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:02.550333 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92536631-57e7-4446-9f52-727013e1d6d4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-92536631-57e7-4446-9f52-727013e1d6d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:02.569697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:02.570591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:02.572048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:02.572048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:02.572048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:02.572048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:02.572048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 241/491] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:02.572048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92536631-57e7-4446-9f52-727013e1d6d4 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:02.600668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:02.615525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe1a7f-4e48-444e-b3e1-5d9715ccf298 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:02.654070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88fb0c-1c0a-4275-b170-f0d67bfecb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:02.654887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88fb0c-1c0a-4275-b170-f0d67bfecb93 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.655337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88fb0c-1c0a-4275-b170-f0d67bfecb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.655756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88fb0c-1c0a-4275-b170-f0d67bfecb93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.667176 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88fb0c-1c0a-4275-b170-f0d67bfecb93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:02.668859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88fb0c-1c0a-4275-b170-f0d67bfecb93 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9f88fb0c-1c0a-4275-b170-f0d67bfecb93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:02.687831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:02.688714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:02.689618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:02.689618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:02.689618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:02.689618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:02.696426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 251/492] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:02.696870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92536631-57e7-4446-9f52-727013e1d6d4 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:02.698150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88fb0c-1c0a-4275-b170-f0d67bfecb93 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:02.718572 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:02.746829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401378de-5691-407f-9f62-3e6be06d6f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:02.747743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401378de-5691-407f-9f62-3e6be06d6f0b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.748100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401378de-5691-407f-9f62-3e6be06d6f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.748450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401378de-5691-407f-9f62-3e6be06d6f0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.764946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401378de-5691-407f-9f62-3e6be06d6f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-401378de-5691-407f-9f62-3e6be06d6f0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:02.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:02.778186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:02.779456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:02.785242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 242/493] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:02.787531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401378de-5691-407f-9f62-3e6be06d6f0b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:02.835084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88fb0c-1c0a-4275-b170-f0d67bfecb93 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:02.874108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d1f19-4e75-4877-8b38-f9974a9e01c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:02.874781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d1f19-4e75-4877-8b38-f9974a9e01c1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.875064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d1f19-4e75-4877-8b38-f9974a9e01c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.875338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d1f19-4e75-4877-8b38-f9974a9e01c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.876065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d1f19-4e75-4877-8b38-f9974a9e01c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:02.876372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d1f19-4e75-4877-8b38-f9974a9e01c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:02.878193 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d1f19-4e75-4877-8b38-f9974a9e01c1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:02.889176 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:02.894489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 252/494] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:02.895307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d1f19-4e75-4877-8b38-f9974a9e01c1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:02.909658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401378de-5691-407f-9f62-3e6be06d6f0b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:02.948875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45d8a7-c735-4020-94ff-fb4fff815559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:02.949494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45d8a7-c735-4020-94ff-fb4fff815559 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.949782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45d8a7-c735-4020-94ff-fb4fff815559 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.950154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45d8a7-c735-4020-94ff-fb4fff815559 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.951199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45d8a7-c735-4020-94ff-fb4fff815559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:02.952006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45d8a7-c735-4020-94ff-fb4fff815559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:02.954678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45d8a7-c735-4020-94ff-fb4fff815559 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:02.973298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 243/495] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:02.974633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e91740-62b6-4174-bfd8-8058f414e966 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:02.974932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e91740-62b6-4174-bfd8-8058f414e966 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:02.976058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e91740-62b6-4174-bfd8-8058f414e966 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:02.980672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 244/496] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:02.980672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e91740-62b6-4174-bfd8-8058f414e966 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:03.018175 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:03.019170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d1f19-4e75-4877-8b38-f9974a9e01c1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:03.062789 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1dcb40-ab5a-4203-8c50-260ccb27ef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:03.063476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dcb40-ab5a-4203-8c50-260ccb27ef25 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.063761 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dcb40-ab5a-4203-8c50-260ccb27ef25 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.064036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dcb40-ab5a-4203-8c50-260ccb27ef25 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.064804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1dcb40-ab5a-4203-8c50-260ccb27ef25 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:03.065231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1dcb40-ab5a-4203-8c50-260ccb27ef25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:03.066938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1dcb40-ab5a-4203-8c50-260ccb27ef25 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:03.077195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 253/497] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:03.078281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1dcb40-ab5a-4203-8c50-260ccb27ef25 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:03.101324 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e91740-62b6-4174-bfd8-8058f414e966 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:03.152724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630bdf3-0f25-4b92-9e1b-c6436e3f958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:03.154015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630bdf3-0f25-4b92-9e1b-c6436e3f958d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.154470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630bdf3-0f25-4b92-9e1b-c6436e3f958d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.154747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630bdf3-0f25-4b92-9e1b-c6436e3f958d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.163125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630bdf3-0f25-4b92-9e1b-c6436e3f958d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:03.163488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:03.164597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630bdf3-0f25-4b92-9e1b-c6436e3f958d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-e630bdf3-0f25-4b92-9e1b-c6436e3f958d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:03.181495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:03.182448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:03.183352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 245/498] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:03.184154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afad295-4455-4b5e-9214-4624baad9ac4 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.184510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afad295-4455-4b5e-9214-4624baad9ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.184794 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afad295-4455-4b5e-9214-4624baad9ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.202408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-5afad295-4455-4b5e-9214-4624baad9ac4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:03.203538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5afad295-4455-4b5e-9214-4624baad9ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:03.213502 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1dcb40-ab5a-4203-8c50-260ccb27ef25 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:03.248051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a79ac6-06be-4f90-baad-47f8e4e4aed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:03.248968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79ac6-06be-4f90-baad-47f8e4e4aed8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.249552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79ac6-06be-4f90-baad-47f8e4e4aed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.249872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79ac6-06be-4f90-baad-47f8e4e4aed8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.258798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a79ac6-06be-4f90-baad-47f8e4e4aed8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:03.260248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a79ac6-06be-4f90-baad-47f8e4e4aed8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:03.273694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 246/499] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:03.274438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afad295-4455-4b5e-9214-4624baad9ac4 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-77a79ac6-06be-4f90-baad-47f8e4e4aed8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:03.275650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:03.276733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:03.278008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 254/500] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jul 14 11:24:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:03.278660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7f4a6-66c9-4285-9fc9-08272dd80a4b None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.278660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7f4a6-66c9-4285-9fc9-08272dd80a4b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.278660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7f4a6-66c9-4285-9fc9-08272dd80a4b None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.279983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 255/501] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:03.281970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7f4a6-66c9-4285-9fc9-08272dd80a4b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:03.306147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:03.378301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afad295-4455-4b5e-9214-4624baad9ac4 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:03.386664 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7f4a6-66c9-4285-9fc9-08272dd80a4b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:03.410010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:03.414789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fccd8-665c-43d4-baf5-748be09de19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:03.415576 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fccd8-665c-43d4-baf5-748be09de19f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.415855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fccd8-665c-43d4-baf5-748be09de19f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.416126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fccd8-665c-43d4-baf5-748be09de19f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.421661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba3d4f-5d16-4a79-8ae4-5e451bee6ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:03.422329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba3d4f-5d16-4a79-8ae4-5e451bee6ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.422617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba3d4f-5d16-4a79-8ae4-5e451bee6ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.423267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba3d4f-5d16-4a79-8ae4-5e451bee6ae5 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.423809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba3d4f-5d16-4a79-8ae4-5e451bee6ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:03.424093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba3d4f-5d16-4a79-8ae4-5e451bee6ae5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:03.425594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba3d4f-5d16-4a79-8ae4-5e451bee6ae5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:03.427547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fccd8-665c-43d4-baf5-748be09de19f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:03.428994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fccd8-665c-43d4-baf5-748be09de19f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-eb1fccd8-665c-43d4-baf5-748be09de19f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:03.448261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:03.449182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:03.450056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:03.450056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:03.450056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:03.450056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:03.450056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 247/502] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:03.450725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fccd8-665c-43d4-baf5-748be09de19f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:03.465757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 256/503] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jul 14 11:24:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:03.466976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6984a-e34c-4549-9663-01b8c36849cf None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.467261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6984a-e34c-4549-9663-01b8c36849cf None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.467539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6984a-e34c-4549-9663-01b8c36849cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.485114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cba6984a-e34c-4549-9663-01b8c36849cf None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:03.486393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cba6984a-e34c-4549-9663-01b8c36849cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:03.533972 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:03.554376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 257/504] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:03.555037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6984a-e34c-4549-9663-01b8c36849cf None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:03.559066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fccd8-665c-43d4-baf5-748be09de19f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:03.593877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a7e87-7347-499d-b224-b31f3e1321ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:03.594538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a7e87-7347-499d-b224-b31f3e1321ff None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.594817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a7e87-7347-499d-b224-b31f3e1321ff None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.595088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a7e87-7347-499d-b224-b31f3e1321ff None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.595932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a7e87-7347-499d-b224-b31f3e1321ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:03.596236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a7e87-7347-499d-b224-b31f3e1321ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:03.597703 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a7e87-7347-499d-b224-b31f3e1321ff None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:03.634066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 248/505] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jul 14 11:24:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:03.634894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a7e87-7347-499d-b224-b31f3e1321ff None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:03.659557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6984a-e34c-4549-9663-01b8c36849cf None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:03.672281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:03.695105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc2849-f640-4643-846b-fb7812237938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:03.695937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc2849-f640-4643-846b-fb7812237938 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.696222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc2849-f640-4643-846b-fb7812237938 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.696493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc2849-f640-4643-846b-fb7812237938 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.706772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc2849-f640-4643-846b-fb7812237938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-bddc2849-f640-4643-846b-fb7812237938 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:03.715289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:03.716363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:03.717201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 258/506] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:03.717880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc2849-f640-4643-846b-fb7812237938 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:03.739230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a7e87-7347-499d-b224-b31f3e1321ff None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:03.773046 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0c6d7-4641-4a45-a5aa-985fec53d83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:03.773729 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0c6d7-4641-4a45-a5aa-985fec53d83d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.774008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0c6d7-4641-4a45-a5aa-985fec53d83d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.774277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0c6d7-4641-4a45-a5aa-985fec53d83d None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.774999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0c6d7-4641-4a45-a5aa-985fec53d83d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:03.775298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0c6d7-4641-4a45-a5aa-985fec53d83d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:03.776679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0c6d7-4641-4a45-a5aa-985fec53d83d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:03.778978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:03.788981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 249/507] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:03.795033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0c6d7-4641-4a45-a5aa-985fec53d83d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:03.818012 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc2849-f640-4643-846b-fb7812237938 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:03.851777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461df355-ba1e-4f3e-b6b9-7a59451c2c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:03.852528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461df355-ba1e-4f3e-b6b9-7a59451c2c15 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.852850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461df355-ba1e-4f3e-b6b9-7a59451c2c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.853129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461df355-ba1e-4f3e-b6b9-7a59451c2c15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.890169 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0c6d7-4641-4a45-a5aa-985fec53d83d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:03.894475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:03.922042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461df355-ba1e-4f3e-b6b9-7a59451c2c15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:03.923611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461df355-ba1e-4f3e-b6b9-7a59451c2c15 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:03.924672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e99b1-755f-4dcd-8c60-4bff29d51c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:03.925500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e99b1-755f-4dcd-8c60-4bff29d51c18 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:03.925782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e99b1-755f-4dcd-8c60-4bff29d51c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:03.926058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e99b1-755f-4dcd-8c60-4bff29d51c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:03.936954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e99b1-755f-4dcd-8c60-4bff29d51c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:03.938475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e99b1-755f-4dcd-8c60-4bff29d51c18 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-bb8e99b1-755f-4dcd-8c60-4bff29d51c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:03.955846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:03.956765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:03.957725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:03.957725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:03.957725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:03.957725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:03.957725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 250/508] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:03.958025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e99b1-755f-4dcd-8c60-4bff29d51c18 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.010736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:04.026279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 259/509] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:04.027127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461df355-ba1e-4f3e-b6b9-7a59451c2c15 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.067217 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e99b1-755f-4dcd-8c60-4bff29d51c18 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:04.093857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:04.101327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41098a7-3c46-4b5c-9d7c-613d1b9ba1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:04.101974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41098a7-3c46-4b5c-9d7c-613d1b9ba1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:04.102252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41098a7-3c46-4b5c-9d7c-613d1b9ba1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:04.102528 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41098a7-3c46-4b5c-9d7c-613d1b9ba1f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:04.122801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461df355-ba1e-4f3e-b6b9-7a59451c2c15 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:04.156956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb6e1f-ff06-40dd-b990-540b7153f35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:04.157669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb6e1f-ff06-40dd-b990-540b7153f35f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:04.157945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb6e1f-ff06-40dd-b990-540b7153f35f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:04.158213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb6e1f-ff06-40dd-b990-540b7153f35f None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:04.158956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb6e1f-ff06-40dd-b990-540b7153f35f None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:04.159256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb6e1f-ff06-40dd-b990-540b7153f35f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:04.160598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb6e1f-ff06-40dd-b990-540b7153f35f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:04.169042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 260/510] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:04.169679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41098a7-3c46-4b5c-9d7c-613d1b9ba1f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:04.169863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb6e1f-ff06-40dd-b990-540b7153f35f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.171294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41098a7-3c46-4b5c-9d7c-613d1b9ba1f7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:04.220301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:04.268154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb6e1f-ff06-40dd-b990-540b7153f35f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:04.272377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 251/511] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:04.273239 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41098a7-3c46-4b5c-9d7c-613d1b9ba1f7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.298859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:04.304288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceec1b0-43e5-471e-9fbb-0455979e0068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:04.305082 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceec1b0-43e5-471e-9fbb-0455979e0068 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:04.305397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceec1b0-43e5-471e-9fbb-0455979e0068 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:04.305672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceec1b0-43e5-471e-9fbb-0455979e0068 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:04.313320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceec1b0-43e5-471e-9fbb-0455979e0068 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:04.314804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceec1b0-43e5-471e-9fbb-0455979e0068 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4ceec1b0-43e5-471e-9fbb-0455979e0068 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:04.329700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:04.330933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:04.332042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 261/512] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:04.332792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceec1b0-43e5-471e-9fbb-0455979e0068 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.374330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41098a7-3c46-4b5c-9d7c-613d1b9ba1f7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:04.408840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508688a-8782-4d14-8858-fe5edaba7801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:04.409527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508688a-8782-4d14-8858-fe5edaba7801 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:04.409817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508688a-8782-4d14-8858-fe5edaba7801 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:04.410091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508688a-8782-4d14-8858-fe5edaba7801 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:04.410809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508688a-8782-4d14-8858-fe5edaba7801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:04.411112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508688a-8782-4d14-8858-fe5edaba7801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:04.412511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508688a-8782-4d14-8858-fe5edaba7801 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:04.423911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 252/513] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:04.424930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508688a-8782-4d14-8858-fe5edaba7801 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.436035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:04.436843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceec1b0-43e5-471e-9fbb-0455979e0068 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:04.471014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd608df-150c-4a56-916c-7b37f81839c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:04.471851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd608df-150c-4a56-916c-7b37f81839c8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:04.472128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd608df-150c-4a56-916c-7b37f81839c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:04.472459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd608df-150c-4a56-916c-7b37f81839c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:04.483009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd608df-150c-4a56-916c-7b37f81839c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:04.484474 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd608df-150c-4a56-916c-7b37f81839c8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-2cd608df-150c-4a56-916c-7b37f81839c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:04.501533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:04.502586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:04.503438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:04.503438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:04.503438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:04.503438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:04.503438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 262/514] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:04.503712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd608df-150c-4a56-916c-7b37f81839c8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.528335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508688a-8782-4d14-8858-fe5edaba7801 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:04.549062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:04.561511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62628046-b09e-4d83-accf-320c40ce29a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:04.562295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62628046-b09e-4d83-accf-320c40ce29a8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:04.562576 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62628046-b09e-4d83-accf-320c40ce29a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:04.562844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62628046-b09e-4d83-accf-320c40ce29a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:04.573823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62628046-b09e-4d83-accf-320c40ce29a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:04.575378 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62628046-b09e-4d83-accf-320c40ce29a8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-62628046-b09e-4d83-accf-320c40ce29a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:04.593300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:04.594192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:04.599623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:04.599623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:04.599623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:04.599623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:04.599623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 253/515] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:04.599623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62628046-b09e-4d83-accf-320c40ce29a8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.604384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd608df-150c-4a56-916c-7b37f81839c8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:04.638312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc671cc-2272-4ace-ba74-35447f95b5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:04.639131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc671cc-2272-4ace-ba74-35447f95b5db None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:04.639386 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc671cc-2272-4ace-ba74-35447f95b5db None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:04.639662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc671cc-2272-4ace-ba74-35447f95b5db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:04.650145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc671cc-2272-4ace-ba74-35447f95b5db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:04.651580 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc671cc-2272-4ace-ba74-35447f95b5db None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:04.652285 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-2dc671cc-2272-4ace-ba74-35447f95b5db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:04.668909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:04.670173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:04.671009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:04.671009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:04.671009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:04.671009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:04.671009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 263/516] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:04.671276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc671cc-2272-4ace-ba74-35447f95b5db None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.697447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62628046-b09e-4d83-accf-320c40ce29a8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:04.733802 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa711c8-2926-42b5-a050-6a34de1d2a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:04.734476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa711c8-2926-42b5-a050-6a34de1d2a30 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:04.734759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa711c8-2926-42b5-a050-6a34de1d2a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:04.735039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa711c8-2926-42b5-a050-6a34de1d2a30 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:04.735894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa711c8-2926-42b5-a050-6a34de1d2a30 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:04.736265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa711c8-2926-42b5-a050-6a34de1d2a30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:04.737796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa711c8-2926-42b5-a050-6a34de1d2a30 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:04.757063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 254/517] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jul 14 11:24:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:04.757953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa711c8-2926-42b5-a050-6a34de1d2a30 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.773519 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:04.779309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc671cc-2272-4ace-ba74-35447f95b5db None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:04.813537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce46c4-ab13-4969-ae4c-cab624421797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:04.814212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce46c4-ab13-4969-ae4c-cab624421797 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:04.814616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce46c4-ab13-4969-ae4c-cab624421797 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:04.815263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce46c4-ab13-4969-ae4c-cab624421797 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:04.815966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce46c4-ab13-4969-ae4c-cab624421797 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:04.817423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce46c4-ab13-4969-ae4c-cab624421797 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:04.855892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa711c8-2926-42b5-a050-6a34de1d2a30 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:04.873989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:04.890903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fb390-d050-4fa7-83d8-67e53cf738a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:04.891565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fb390-d050-4fa7-83d8-67e53cf738a2 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:04.891848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fb390-d050-4fa7-83d8-67e53cf738a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:04.892124 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fb390-d050-4fa7-83d8-67e53cf738a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:04.892842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fb390-d050-4fa7-83d8-67e53cf738a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:04.893145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fb390-d050-4fa7-83d8-67e53cf738a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:04.894559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fb390-d050-4fa7-83d8-67e53cf738a2 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:04.905592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 255/518] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:04.906386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fb390-d050-4fa7-83d8-67e53cf738a2 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:04.945468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 264/519] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:24:04 2022] POST /identity/v3/users => generated 247 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:04.946280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce46c4-ab13-4969-ae4c-cab624421797 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.017753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fb390-d050-4fa7-83d8-67e53cf738a2 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.055805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb0076-3c90-4bca-931c-6ff544189cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.056389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb0076-3c90-4bca-931c-6ff544189cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.056675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb0076-3c90-4bca-931c-6ff544189cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.056955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb0076-3c90-4bca-931c-6ff544189cb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.057735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb0076-3c90-4bca-931c-6ff544189cb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:05.058042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb0076-3c90-4bca-931c-6ff544189cb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.059445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb0076-3c90-4bca-931c-6ff544189cb6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.066566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce46c4-ab13-4969-ae4c-cab624421797 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.073682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 256/520] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:05.074547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb0076-3c90-4bca-931c-6ff544189cb6 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.111866 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ec5c4-170b-4531-809f-726817717873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.112567 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ec5c4-170b-4531-809f-726817717873 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.112815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ec5c4-170b-4531-809f-726817717873 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.113089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ec5c4-170b-4531-809f-726817717873 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.113899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ec5c4-170b-4531-809f-726817717873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:05.114205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ec5c4-170b-4531-809f-726817717873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.116140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ec5c4-170b-4531-809f-726817717873 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.121104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:05.130047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 265/521] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:05.131223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ec5c4-170b-4531-809f-726817717873 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.147312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:05.189935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb0076-3c90-4bca-931c-6ff544189cb6 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.227469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdbbc1-0b5d-42fe-b18b-9083e5547827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.228112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdbbc1-0b5d-42fe-b18b-9083e5547827 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.228396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdbbc1-0b5d-42fe-b18b-9083e5547827 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.228669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdbbc1-0b5d-42fe-b18b-9083e5547827 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.241453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ec5c4-170b-4531-809f-726817717873 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.279451 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5e2b2-47f5-4288-a9da-3d38ac8f1aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.280251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5e2b2-47f5-4288-a9da-3d38ac8f1aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.280532 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5e2b2-47f5-4288-a9da-3d38ac8f1aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.280803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5e2b2-47f5-4288-a9da-3d38ac8f1aa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.287706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5e2b2-47f5-4288-a9da-3d38ac8f1aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.289219 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5e2b2-47f5-4288-a9da-3d38ac8f1aa3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.301887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdbbc1-0b5d-42fe-b18b-9083e5547827 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.303342 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdbbc1-0b5d-42fe-b18b-9083e5547827 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-97c5e2b2-47f5-4288-a9da-3d38ac8f1aa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:05.304068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:05.304929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:05.305873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:05.305873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:05.305873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:05.305873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:05.305873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:05.305873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:05.305873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:05.305873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:05.305873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 266/522] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:05.306999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5e2b2-47f5-4288-a9da-3d38ac8f1aa3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.350396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:05.379046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:05.409911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5e2b2-47f5-4288-a9da-3d38ac8f1aa3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.420157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 257/523] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:05.420157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdbbc1-0b5d-42fe-b18b-9083e5547827 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.445088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fec50-f49f-4cc2-ab10-a54427a05606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.446016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fec50-f49f-4cc2-ab10-a54427a05606 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.446299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fec50-f49f-4cc2-ab10-a54427a05606 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.446572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fec50-f49f-4cc2-ab10-a54427a05606 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.455764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fec50-f49f-4cc2-ab10-a54427a05606 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.457258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fec50-f49f-4cc2-ab10-a54427a05606 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-ca2fec50-f49f-4cc2-ab10-a54427a05606 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:05.471522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:05.472542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:05.473473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 267/524] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:05.474681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fec50-f49f-4cc2-ab10-a54427a05606 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.542562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdbbc1-0b5d-42fe-b18b-9083e5547827 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.578914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fec50-f49f-4cc2-ab10-a54427a05606 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.579884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5021707-186d-474a-b1a5-2da6f495fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.580668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5021707-186d-474a-b1a5-2da6f495fac7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.580945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5021707-186d-474a-b1a5-2da6f495fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.581939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5021707-186d-474a-b1a5-2da6f495fac7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.588866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5021707-186d-474a-b1a5-2da6f495fac7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.590464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5021707-186d-474a-b1a5-2da6f495fac7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.592328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-f5021707-186d-474a-b1a5-2da6f495fac7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:05.606133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:05.612615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:05.613485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:05.613485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:05.613485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:05.613485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:05.613485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:05.613485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:05.613485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:05.613485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:05.613485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 258/525] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:05.613485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5021707-186d-474a-b1a5-2da6f495fac7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.615890 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:05.620696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145176db-afef-4114-a0e0-8568af19a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.621390 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145176db-afef-4114-a0e0-8568af19a67e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.621679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145176db-afef-4114-a0e0-8568af19a67e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.621959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145176db-afef-4114-a0e0-8568af19a67e None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.622755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145176db-afef-4114-a0e0-8568af19a67e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:05.623068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145176db-afef-4114-a0e0-8568af19a67e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.624502 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145176db-afef-4114-a0e0-8568af19a67e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.647679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 268/526] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:05.647679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145176db-afef-4114-a0e0-8568af19a67e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.723484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5021707-186d-474a-b1a5-2da6f495fac7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.748628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145176db-afef-4114-a0e0-8568af19a67e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.759790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c49818-2a78-4856-857a-c928abb2233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.760577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c49818-2a78-4856-857a-c928abb2233a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.760863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c49818-2a78-4856-857a-c928abb2233a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.761143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c49818-2a78-4856-857a-c928abb2233a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.774164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c49818-2a78-4856-857a-c928abb2233a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.776178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c49818-2a78-4856-857a-c928abb2233a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.784153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99decc-f377-460a-a51e-b14605745387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.784819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99decc-f377-460a-a51e-b14605745387 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.785098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99decc-f377-460a-a51e-b14605745387 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.785417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99decc-f377-460a-a51e-b14605745387 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.786278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99decc-f377-460a-a51e-b14605745387 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:05.786583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99decc-f377-460a-a51e-b14605745387 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.787986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99decc-f377-460a-a51e-b14605745387 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-d1c49818-2a78-4856-857a-c928abb2233a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:05.802876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:05.803705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:05.810076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:05.810076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:05.810076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:05.810076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:05.810076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 259/527] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:05.810076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c49818-2a78-4856-857a-c928abb2233a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.818864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 269/528] 10.222.0.40 () {56 vars in 1139 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:05.818864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99decc-f377-460a-a51e-b14605745387 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.868373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:05.876573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:05.914719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c49818-2a78-4856-857a-c928abb2233a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.926652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99decc-f377-460a-a51e-b14605745387 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:05.953693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80e38-25cb-46ea-8a98-1bb27fb73b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.954650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80e38-25cb-46ea-8a98-1bb27fb73b54 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.954940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80e38-25cb-46ea-8a98-1bb27fb73b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.955372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80e38-25cb-46ea-8a98-1bb27fb73b54 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.956498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e80e38-25cb-46ea-8a98-1bb27fb73b54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:05.956967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e80e38-25cb-46ea-8a98-1bb27fb73b54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.959024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e80e38-25cb-46ea-8a98-1bb27fb73b54 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.962185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db8863-ab8b-48f1-91c0-8227cfaabbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:05.962858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db8863-ab8b-48f1-91c0-8227cfaabbcb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:05.963511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db8863-ab8b-48f1-91c0-8227cfaabbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:05.963788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db8863-ab8b-48f1-91c0-8227cfaabbcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:05.964620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db8863-ab8b-48f1-91c0-8227cfaabbcb None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=a1a7213cbf6448b7ba6570a52a4c7c0e) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:05.964927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db8863-ab8b-48f1-91c0-8227cfaabbcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:05.966412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db8863-ab8b-48f1-91c0-8227cfaabbcb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:05.978653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 270/529] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/role_assignments?user.id=a1a7213cbf6448b7ba6570a52a4c7c0e&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:05.978653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db8863-ab8b-48f1-91c0-8227cfaabbcb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:05.978898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 260/530] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:05.978898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80e38-25cb-46ea-8a98-1bb27fb73b54 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:06.102482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80e38-25cb-46ea-8a98-1bb27fb73b54 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:06.120200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:06.135392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db8863-ab8b-48f1-91c0-8227cfaabbcb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:06.151278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:06.156376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22dd8f-cd35-4e89-adbe-a2e2f8e8da02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:06.157386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22dd8f-cd35-4e89-adbe-a2e2f8e8da02 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:06.157879 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22dd8f-cd35-4e89-adbe-a2e2f8e8da02 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:06.158376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22dd8f-cd35-4e89-adbe-a2e2f8e8da02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:06.175058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430afc4d-9fe5-4a88-81c8-c1f9d3cd90d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:06.175726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430afc4d-9fe5-4a88-81c8-c1f9d3cd90d5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:06.182558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430afc4d-9fe5-4a88-81c8-c1f9d3cd90d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:06.182558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430afc4d-9fe5-4a88-81c8-c1f9d3cd90d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:06.182558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430afc4d-9fe5-4a88-81c8-c1f9d3cd90d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:06.182558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430afc4d-9fe5-4a88-81c8-c1f9d3cd90d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:06.182558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430afc4d-9fe5-4a88-81c8-c1f9d3cd90d5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:06.195845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 271/531] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:06.197239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430afc4d-9fe5-4a88-81c8-c1f9d3cd90d5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:06.249669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22dd8f-cd35-4e89-adbe-a2e2f8e8da02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:06.251674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22dd8f-cd35-4e89-adbe-a2e2f8e8da02 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:06.308379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430afc4d-9fe5-4a88-81c8-c1f9d3cd90d5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:06.344882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20067fd9-4d68-431e-86d9-0cd29f332ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:06.345926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20067fd9-4d68-431e-86d9-0cd29f332ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:06.346412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20067fd9-4d68-431e-86d9-0cd29f332ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:06.346889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20067fd9-4d68-431e-86d9-0cd29f332ad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:06.347940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20067fd9-4d68-431e-86d9-0cd29f332ad4 None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=147d764ef17042dbb7ea179c29711588) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:06.348490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20067fd9-4d68-431e-86d9-0cd29f332ad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:06.350163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20067fd9-4d68-431e-86d9-0cd29f332ad4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:06.361613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 272/533] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:06 2022] GET /identity/v3/role_assignments?user.id=147d764ef17042dbb7ea179c29711588&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:06.362915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20067fd9-4d68-431e-86d9-0cd29f332ad4 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:06.369183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 261/533] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:06.369183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22dd8f-cd35-4e89-adbe-a2e2f8e8da02 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:06.389649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:06.446099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:06.515533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20067fd9-4d68-431e-86d9-0cd29f332ad4 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:06.543402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22dd8f-cd35-4e89-adbe-a2e2f8e8da02 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:06.553834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca50e54-8483-4f7c-becf-8b3528ef9b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:06.554642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca50e54-8483-4f7c-becf-8b3528ef9b2e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:06.554926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca50e54-8483-4f7c-becf-8b3528ef9b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:06.555200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca50e54-8483-4f7c-becf-8b3528ef9b2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:06.562739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca50e54-8483-4f7c-becf-8b3528ef9b2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:06.564211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca50e54-8483-4f7c-becf-8b3528ef9b2e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-eca50e54-8483-4f7c-becf-8b3528ef9b2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:06.579108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:06.580009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:06.580843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:06.580843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:06.580843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:06.580843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:06.580843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:06.580843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:06.580843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:06.580843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:06.580843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 273/534] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:06.588705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca50e54-8483-4f7c-becf-8b3528ef9b2e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:06.619689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f949cb0-cd0c-46f9-bbca-bdf6c04e3bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:06.620868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f949cb0-cd0c-46f9-bbca-bdf6c04e3bbf None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:06.621469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f949cb0-cd0c-46f9-bbca-bdf6c04e3bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:06.621935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f949cb0-cd0c-46f9-bbca-bdf6c04e3bbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:06.637392 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f949cb0-cd0c-46f9-bbca-bdf6c04e3bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:06.639877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f949cb0-cd0c-46f9-bbca-bdf6c04e3bbf None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-2f949cb0-cd0c-46f9-bbca-bdf6c04e3bbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:06.664345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:06.665384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:06.666228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:06.666228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:06.666228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:06.666228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:06.666228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 262/535] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:06.671387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f949cb0-cd0c-46f9-bbca-bdf6c04e3bbf None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:06.747220 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:06.771861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:06.779936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca50e54-8483-4f7c-becf-8b3528ef9b2e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:06.795461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f949cb0-cd0c-46f9-bbca-bdf6c04e3bbf None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:06.829660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ede7cb-1804-4684-bf26-757780c24630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:06.830333 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ede7cb-1804-4684-bf26-757780c24630 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:06.830623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ede7cb-1804-4684-bf26-757780c24630 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:06.830904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ede7cb-1804-4684-bf26-757780c24630 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:06.831646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ede7cb-1804-4684-bf26-757780c24630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:06.831956 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ede7cb-1804-4684-bf26-757780c24630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:06.834516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ede7cb-1804-4684-bf26-757780c24630 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:06.848215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 263/536] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:06.854479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ede7cb-1804-4684-bf26-757780c24630 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:06.895429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40ecd4-8394-42aa-b9c7-eaed061c03a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:06.896574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40ecd4-8394-42aa-b9c7-eaed061c03a6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:06.896854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40ecd4-8394-42aa-b9c7-eaed061c03a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:06.897186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40ecd4-8394-42aa-b9c7-eaed061c03a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:06.897999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40ecd4-8394-42aa-b9c7-eaed061c03a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:06.898303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40ecd4-8394-42aa-b9c7-eaed061c03a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:06.899786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40ecd4-8394-42aa-b9c7-eaed061c03a6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:06.931508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 274/537] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:06.940142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40ecd4-8394-42aa-b9c7-eaed061c03a6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:06.969232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ede7cb-1804-4684-bf26-757780c24630 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:07.006017 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:07.017917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c45da9-012e-4292-991e-af59f120772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:07.018739 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c45da9-012e-4292-991e-af59f120772b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:07.019029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c45da9-012e-4292-991e-af59f120772b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:07.019307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c45da9-012e-4292-991e-af59f120772b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:07.034386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c45da9-012e-4292-991e-af59f120772b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:07.035876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c45da9-012e-4292-991e-af59f120772b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-a7c45da9-012e-4292-991e-af59f120772b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:07.075275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:07.076472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:07.077772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:07.077772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:07.077772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:07.077772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:07.077772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:07.077772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:07.077772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:07.077772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:07.078361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 264/538] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:07.086231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c45da9-012e-4292-991e-af59f120772b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:07.108659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40ecd4-8394-42aa-b9c7-eaed061c03a6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:07.154976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a750e33-3279-43b8-9f2c-f9eb80100138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:07.155848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a750e33-3279-43b8-9f2c-f9eb80100138 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:07.156132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a750e33-3279-43b8-9f2c-f9eb80100138 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:07.156410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a750e33-3279-43b8-9f2c-f9eb80100138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:07.157563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a750e33-3279-43b8-9f2c-f9eb80100138 None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=f12e348368ac42c696623b4a8f781977) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:07.158294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a750e33-3279-43b8-9f2c-f9eb80100138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:07.159785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a750e33-3279-43b8-9f2c-f9eb80100138 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:07.168753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 275/539] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:06 2022] GET /identity/v3/role_assignments?user.id=f12e348368ac42c696623b4a8f781977&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:07.170172 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6d579-4c3a-4e17-9086-8eda27d4f874 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:07.170460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6d579-4c3a-4e17-9086-8eda27d4f874 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:07.170735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6d579-4c3a-4e17-9086-8eda27d4f874 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:07.172326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 276/540] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:07.178653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51f3c3-9fb7-4b46-bf5d-68bb50a41d82 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:07.179345 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51f3c3-9fb7-4b46-bf5d-68bb50a41d82 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:07.179857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51f3c3-9fb7-4b46-bf5d-68bb50a41d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:07.199949 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ae51f3c3-9fb7-4b46-bf5d-68bb50a41d82 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:07.201828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ae51f3c3-9fb7-4b46-bf5d-68bb50a41d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:07.228180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:07.283771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c45da9-012e-4292-991e-af59f120772b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:07.297257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 277/541] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:07.343750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51f3c3-9fb7-4b46-bf5d-68bb50a41d82 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:07.369981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3022f6-58ee-4144-8ae3-7c7ffcfd1d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:07.370900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3022f6-58ee-4144-8ae3-7c7ffcfd1d46 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:07.371358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3022f6-58ee-4144-8ae3-7c7ffcfd1d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:07.371794 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3022f6-58ee-4144-8ae3-7c7ffcfd1d46 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:07.372867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3022f6-58ee-4144-8ae3-7c7ffcfd1d46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:07.374218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3022f6-58ee-4144-8ae3-7c7ffcfd1d46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:07.375692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3022f6-58ee-4144-8ae3-7c7ffcfd1d46 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:07.389358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 265/542] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:07.395105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3022f6-58ee-4144-8ae3-7c7ffcfd1d46 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:07.452341 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:07.536033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51f3c3-9fb7-4b46-bf5d-68bb50a41d82 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:07.555356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3022f6-58ee-4144-8ae3-7c7ffcfd1d46 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:07.601865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a97e2-b6b6-484a-9ce5-49ba63ee1dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:07.602672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a97e2-b6b6-484a-9ce5-49ba63ee1dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:07.602952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a97e2-b6b6-484a-9ce5-49ba63ee1dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:07.603224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a97e2-b6b6-484a-9ce5-49ba63ee1dd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:07.618468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515da7b9-8661-416c-b302-b72fd8c95f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:07.619135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515da7b9-8661-416c-b302-b72fd8c95f6f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:07.619414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515da7b9-8661-416c-b302-b72fd8c95f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:07.619691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515da7b9-8661-416c-b302-b72fd8c95f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:07.620545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515da7b9-8661-416c-b302-b72fd8c95f6f None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=481ef56b1cfa465a8efc59736a34e412) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:07.620854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515da7b9-8661-416c-b302-b72fd8c95f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:07.622335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515da7b9-8661-416c-b302-b72fd8c95f6f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:07.628370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a97e2-b6b6-484a-9ce5-49ba63ee1dd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:07.639348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 266/543] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:07 2022] GET /identity/v3/role_assignments?user.id=481ef56b1cfa465a8efc59736a34e412&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-d00a97e2-b6b6-484a-9ce5-49ba63ee1dd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:07.640279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:07.641150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:07.641969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:07.645320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 278/544] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:07.648712 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515da7b9-8661-416c-b302-b72fd8c95f6f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:07.824394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515da7b9-8661-416c-b302-b72fd8c95f6f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:07.833785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:07.866523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a811b6b-149f-4a50-a8c5-560630713bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:07.867202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a811b6b-149f-4a50-a8c5-560630713bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:07.867481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a811b6b-149f-4a50-a8c5-560630713bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:07.867758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a811b6b-149f-4a50-a8c5-560630713bd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:07.868512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a811b6b-149f-4a50-a8c5-560630713bd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:07.868815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a811b6b-149f-4a50-a8c5-560630713bd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:07.870329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a811b6b-149f-4a50-a8c5-560630713bd5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:07.879271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 267/545] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:07.885832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a97e2-b6b6-484a-9ce5-49ba63ee1dd0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:07.912003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:08.022539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a97e2-b6b6-484a-9ce5-49ba63ee1dd0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:08.064784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10774cd1-6eb7-4f8f-b41e-b99b8a558379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:08.065538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774cd1-6eb7-4f8f-b41e-b99b8a558379 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:08.065994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774cd1-6eb7-4f8f-b41e-b99b8a558379 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:08.066277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774cd1-6eb7-4f8f-b41e-b99b8a558379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:08.078085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10774cd1-6eb7-4f8f-b41e-b99b8a558379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:08.079544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10774cd1-6eb7-4f8f-b41e-b99b8a558379 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-10774cd1-6eb7-4f8f-b41e-b99b8a558379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:08.099341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:08.100238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:08.101642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:08.101642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:08.101642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:08.101642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:08.105397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 279/546] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:08.110413 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a811b6b-149f-4a50-a8c5-560630713bd5 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:08.149733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3bf7f-3ca8-43e4-a461-a8fe1c5a22d6 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:08.150446 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3bf7f-3ca8-43e4-a461-a8fe1c5a22d6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:08.150939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3bf7f-3ca8-43e4-a461-a8fe1c5a22d6 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:08.152861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 280/547] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:24:08.158426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75b43e-5cfc-458d-906c-554ce7a7b1ee None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:08.159189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75b43e-5cfc-458d-906c-554ce7a7b1ee None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:08.159913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75b43e-5cfc-458d-906c-554ce7a7b1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:08.179291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3a75b43e-5cfc-458d-906c-554ce7a7b1ee None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:08.180813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3a75b43e-5cfc-458d-906c-554ce7a7b1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:08.265514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:08.278923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 281/548] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:24:08.305304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a811b6b-149f-4a50-a8c5-560630713bd5 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:08.312568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75b43e-5cfc-458d-906c-554ce7a7b1ee None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:08.371340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e011782-15e6-4243-a7bb-35269bc30a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:08.371923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011782-15e6-4243-a7bb-35269bc30a2d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:08.372203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011782-15e6-4243-a7bb-35269bc30a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:08.372475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011782-15e6-4243-a7bb-35269bc30a2d None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:08.373220 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e011782-15e6-4243-a7bb-35269bc30a2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:08.373525 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e011782-15e6-4243-a7bb-35269bc30a2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:08.375672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e011782-15e6-4243-a7bb-35269bc30a2d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:08.398992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 268/549] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:08.406196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e011782-15e6-4243-a7bb-35269bc30a2d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:08.455059 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:08.470015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75b43e-5cfc-458d-906c-554ce7a7b1ee None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:08.532399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d405f-678d-4a8e-9768-772d6eba66fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:08.533276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d405f-678d-4a8e-9768-772d6eba66fb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:08.533564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d405f-678d-4a8e-9768-772d6eba66fb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:08.533831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d405f-678d-4a8e-9768-772d6eba66fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:08.561085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d405f-678d-4a8e-9768-772d6eba66fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-007d405f-678d-4a8e-9768-772d6eba66fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:08.577612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:08.578675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:08.579586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:08.585203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 282/550] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:08.588554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d405f-678d-4a8e-9768-772d6eba66fb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:08.596928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e011782-15e6-4243-a7bb-35269bc30a2d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:08.664495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc1ec-d8d2-47e5-9eed-b5a2ed0c6baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:08.665229 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc1ec-d8d2-47e5-9eed-b5a2ed0c6baf None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:08.665518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc1ec-d8d2-47e5-9eed-b5a2ed0c6baf None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:08.665790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc1ec-d8d2-47e5-9eed-b5a2ed0c6baf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:08.680292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fc1ec-d8d2-47e5-9eed-b5a2ed0c6baf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:08.682637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fc1ec-d8d2-47e5-9eed-b5a2ed0c6baf None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:08.694872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-314fc1ec-d8d2-47e5-9eed-b5a2ed0c6baf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:08.714613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:08.715583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:08.716504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 269/551] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:08.720713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc1ec-d8d2-47e5-9eed-b5a2ed0c6baf None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:08.786119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d405f-678d-4a8e-9768-772d6eba66fb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:08.862012 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2d1ed-cc47-4ac7-9696-05897e26599c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:08.865392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2d1ed-cc47-4ac7-9696-05897e26599c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:08.865898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2d1ed-cc47-4ac7-9696-05897e26599c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:08.866401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2d1ed-cc47-4ac7-9696-05897e26599c None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:08.867377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2d1ed-cc47-4ac7-9696-05897e26599c None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:08.869547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2d1ed-cc47-4ac7-9696-05897e26599c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:08.871175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2d1ed-cc47-4ac7-9696-05897e26599c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:08.894806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 283/552] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:08.897102 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2d1ed-cc47-4ac7-9696-05897e26599c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:08.930464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:08.954858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc1ec-d8d2-47e5-9eed-b5a2ed0c6baf None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:09.055164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11948f4-8ec9-41ab-9ec1-0b815e77161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:09.056252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11948f4-8ec9-41ab-9ec1-0b815e77161e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.056787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11948f4-8ec9-41ab-9ec1-0b815e77161e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.057313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11948f4-8ec9-41ab-9ec1-0b815e77161e None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.058610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11948f4-8ec9-41ab-9ec1-0b815e77161e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:09.060478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11948f4-8ec9-41ab-9ec1-0b815e77161e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:09.062222 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11948f4-8ec9-41ab-9ec1-0b815e77161e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:09.096073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2d1ed-cc47-4ac7-9696-05897e26599c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:09.108412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 270/553] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jul 14 11:24:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:09.113604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11948f4-8ec9-41ab-9ec1-0b815e77161e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:09.122044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:09.146485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c3bcd-e722-4d1e-9e05-2f29d0dbd9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:09.148053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c3bcd-e722-4d1e-9e05-2f29d0dbd9be None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.148820 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c3bcd-e722-4d1e-9e05-2f29d0dbd9be None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.149787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c3bcd-e722-4d1e-9e05-2f29d0dbd9be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.167112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c3bcd-e722-4d1e-9e05-2f29d0dbd9be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:09.171185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c3bcd-e722-4d1e-9e05-2f29d0dbd9be None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-e63c3bcd-e722-4d1e-9e05-2f29d0dbd9be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:09.190872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:09.191797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:09.193375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:09.193375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:09.193375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:09.193375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:09.195653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 284/554] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:09.197231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f6ae1-c599-4d0d-8fe5-8b100fdf7742 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.197764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f6ae1-c599-4d0d-8fe5-8b100fdf7742 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.198252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f6ae1-c599-4d0d-8fe5-8b100fdf7742 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.199977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 285/555] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:09.200886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f6ae1-c599-4d0d-8fe5-8b100fdf7742 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:09.241541 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11948f4-8ec9-41ab-9ec1-0b815e77161e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:09.281284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fe40b-2d3c-40b2-8968-9957d6034e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:09.281932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fe40b-2d3c-40b2-8968-9957d6034e4f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.282212 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fe40b-2d3c-40b2-8968-9957d6034e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.282483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fe40b-2d3c-40b2-8968-9957d6034e4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.303451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:09.312653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f6ae1-c599-4d0d-8fe5-8b100fdf7742 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:09.349681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff960b-4984-4d9a-a09b-143c77341f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:09.350732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff960b-4984-4d9a-a09b-143c77341f13 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.351218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff960b-4984-4d9a-a09b-143c77341f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.352138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff960b-4984-4d9a-a09b-143c77341f13 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.353387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff960b-4984-4d9a-a09b-143c77341f13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:09.353906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff960b-4984-4d9a-a09b-143c77341f13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:09.356141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff960b-4984-4d9a-a09b-143c77341f13 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:09.369860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fe40b-2d3c-40b2-8968-9957d6034e4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:09.372328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 286/556] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:09.373879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b05d-8997-4fc7-9fb1-efd5b4bc5df7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.374338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b05d-8997-4fc7-9fb1-efd5b4bc5df7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.374800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b05d-8997-4fc7-9fb1-efd5b4bc5df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.376806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fe40b-2d3c-40b2-8968-9957d6034e4f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:09.400894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-70c2b05d-8997-4fc7-9fb1-efd5b4bc5df7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:09.402733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-70c2b05d-8997-4fc7-9fb1-efd5b4bc5df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:09.436734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:09.496069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 287/557] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:09.497237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2b05d-8997-4fc7-9fb1-efd5b4bc5df7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:09.501756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 271/558] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 11:24:09.502759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fe40b-2d3c-40b2-8968-9957d6034e4f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:09.568411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:09.612191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2b05d-8997-4fc7-9fb1-efd5b4bc5df7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:09.625520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fe40b-2d3c-40b2-8968-9957d6034e4f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:09.648303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52206ff0-1ed3-43e7-aa32-573f7e130d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:09.649100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52206ff0-1ed3-43e7-aa32-573f7e130d96 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.649411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52206ff0-1ed3-43e7-aa32-573f7e130d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.649682 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52206ff0-1ed3-43e7-aa32-573f7e130d96 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.656929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52206ff0-1ed3-43e7-aa32-573f7e130d96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:09.658411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52206ff0-1ed3-43e7-aa32-573f7e130d96 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:09.661809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06850de4-9f7e-4176-9126-8d90203ffa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:09.662592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06850de4-9f7e-4176-9126-8d90203ffa6f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.662870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06850de4-9f7e-4176-9126-8d90203ffa6f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.663140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06850de4-9f7e-4176-9126-8d90203ffa6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-52206ff0-1ed3-43e7-aa32-573f7e130d96 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:09.674338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:09.675430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:09.676412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:09.676412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:09.676412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:09.676412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:09.676412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:09.676412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:09.676412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:09.676412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:09.676412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 11:24:09.676412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:09.676991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06850de4-9f7e-4176-9126-8d90203ffa6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:09.677067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 288/559] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:09.677683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62543c2a-2560-4326-b5ff-4c99be40214b None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.677683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62543c2a-2560-4326-b5ff-4c99be40214b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.677842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62543c2a-2560-4326-b5ff-4c99be40214b None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.679164 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 289/560] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:09.681000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06850de4-9f7e-4176-9126-8d90203ffa6f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:09.681771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62543c2a-2560-4326-b5ff-4c99be40214b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-06850de4-9f7e-4176-9126-8d90203ffa6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:09.700606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:09.702122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:09.703016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:09.703016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:09.703016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:09.703016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:09.703016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 272/561] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:09.703430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06850de4-9f7e-4176-9126-8d90203ffa6f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:09.711429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:09.807698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:09.811507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06850de4-9f7e-4176-9126-8d90203ffa6f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:09.818852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62543c2a-2560-4326-b5ff-4c99be40214b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:09.851405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee484392-d14c-49a3-8c88-a8d255b04f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:09.852082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee484392-d14c-49a3-8c88-a8d255b04f14 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.852362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee484392-d14c-49a3-8c88-a8d255b04f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.852638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee484392-d14c-49a3-8c88-a8d255b04f14 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.853527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee484392-d14c-49a3-8c88-a8d255b04f14 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:09.853831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee484392-d14c-49a3-8c88-a8d255b04f14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:09.855212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033f36f-8f93-49d2-b119-f664994b4a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:09.855348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee484392-d14c-49a3-8c88-a8d255b04f14 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:09.856119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033f36f-8f93-49d2-b119-f664994b4a48 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.856405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033f36f-8f93-49d2-b119-f664994b4a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.856683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033f36f-8f93-49d2-b119-f664994b4a48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.873887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033f36f-8f93-49d2-b119-f664994b4a48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:09.879584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 273/562] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jul 14 11:24:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:09.879584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61eb62-1eb2-4596-9a89-6802d849b379 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:09.879584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61eb62-1eb2-4596-9a89-6802d849b379 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:09.879584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61eb62-1eb2-4596-9a89-6802d849b379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-a033f36f-8f93-49d2-b119-f664994b4a48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:09.882845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:09.883954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:09.884776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 290/563] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:09.885571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033f36f-8f93-49d2-b119-f664994b4a48 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:09.899814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ad61eb62-1eb2-4596-9a89-6802d849b379 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:09.901028 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ad61eb62-1eb2-4596-9a89-6802d849b379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:09.967462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:09.969581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 274/564] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:09.970431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61eb62-1eb2-4596-9a89-6802d849b379 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:09.993831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033f36f-8f93-49d2-b119-f664994b4a48 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.027526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccbb35-6d51-4148-8332-e263c3604536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.028185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccbb35-6d51-4148-8332-e263c3604536 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.028462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccbb35-6d51-4148-8332-e263c3604536 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.028733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccbb35-6d51-4148-8332-e263c3604536 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.029500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccbb35-6d51-4148-8332-e263c3604536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:10.029805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccbb35-6d51-4148-8332-e263c3604536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:10.031181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccbb35-6d51-4148-8332-e263c3604536 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:10.048692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 291/565] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:10.049840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccbb35-6d51-4148-8332-e263c3604536 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.071508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:10.087348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61eb62-1eb2-4596-9a89-6802d849b379 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.121580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fea558-929f-4d31-bce5-a2bcadcbc3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.122366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fea558-929f-4d31-bce5-a2bcadcbc3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.122645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fea558-929f-4d31-bce5-a2bcadcbc3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.122913 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fea558-929f-4d31-bce5-a2bcadcbc3a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.133064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fea558-929f-4d31-bce5-a2bcadcbc3a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:10.134549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fea558-929f-4d31-bce5-a2bcadcbc3a4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-e0fea558-929f-4d31-bce5-a2bcadcbc3a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:10.152052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:10.153121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:10.154258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:10.154258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:10.154258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:10.154258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:10.154258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 275/566] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:10.154566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29445-8422-4c93-9b61-85b4b2718f7e None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.154707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29445-8422-4c93-9b61-85b4b2718f7e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.154948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29445-8422-4c93-9b61-85b4b2718f7e None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.156339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 276/567] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:10.156858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab29445-8422-4c93-9b61-85b4b2718f7e None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.197141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccbb35-6d51-4148-8332-e263c3604536 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.231547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23176e7-d000-4bf8-834f-b96129f69202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.232135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23176e7-d000-4bf8-834f-b96129f69202 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.232414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23176e7-d000-4bf8-834f-b96129f69202 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.232691 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23176e7-d000-4bf8-834f-b96129f69202 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.233474 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23176e7-d000-4bf8-834f-b96129f69202 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:10.233782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23176e7-d000-4bf8-834f-b96129f69202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:10.235140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23176e7-d000-4bf8-834f-b96129f69202 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:10.240459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:10.248108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 292/568] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:10.248108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23176e7-d000-4bf8-834f-b96129f69202 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.264107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab29445-8422-4c93-9b61-85b4b2718f7e None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.301765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84df90f-bdd8-4c79-b5a6-370dd75c8a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.302561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84df90f-bdd8-4c79-b5a6-370dd75c8a0d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.302839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84df90f-bdd8-4c79-b5a6-370dd75c8a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.303114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84df90f-bdd8-4c79-b5a6-370dd75c8a0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.309961 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:10.313796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84df90f-bdd8-4c79-b5a6-370dd75c8a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-e84df90f-bdd8-4c79-b5a6-370dd75c8a0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:10.331842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:10.332880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 277/569] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:10.333790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84df90f-bdd8-4c79-b5a6-370dd75c8a0d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.347315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23176e7-d000-4bf8-834f-b96129f69202 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.380233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67e40a-2e8e-4f2a-8836-44fb27d3de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.381027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67e40a-2e8e-4f2a-8836-44fb27d3de59 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.381344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67e40a-2e8e-4f2a-8836-44fb27d3de59 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.381623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67e40a-2e8e-4f2a-8836-44fb27d3de59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.388501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67e40a-2e8e-4f2a-8836-44fb27d3de59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:10.390031 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67e40a-2e8e-4f2a-8836-44fb27d3de59 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-2d67e40a-2e8e-4f2a-8836-44fb27d3de59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:10.405047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:10.406037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:10.406915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:10.406915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:10.406915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:10.406915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:10.406915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:10.406915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:10.406915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:10.406915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:10.406915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 293/570] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:10.407597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad1af3-2252-433f-9baf-b253d513a1eb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.407897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad1af3-2252-433f-9baf-b253d513a1eb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.408146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad1af3-2252-433f-9baf-b253d513a1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.424733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-79ad1af3-2252-433f-9baf-b253d513a1eb None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:10.425974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-79ad1af3-2252-433f-9baf-b253d513a1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:10.434813 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84df90f-bdd8-4c79-b5a6-370dd75c8a0d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.471793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f954886-6679-46b0-96ec-f05b81f2ee42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.472447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f954886-6679-46b0-96ec-f05b81f2ee42 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.472734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f954886-6679-46b0-96ec-f05b81f2ee42 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.473007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f954886-6679-46b0-96ec-f05b81f2ee42 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.473803 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f954886-6679-46b0-96ec-f05b81f2ee42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:10.474108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f954886-6679-46b0-96ec-f05b81f2ee42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:10.475492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f954886-6679-46b0-96ec-f05b81f2ee42 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:10.482882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:10.487121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 278/571] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:10.487911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f954886-6679-46b0-96ec-f05b81f2ee42 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.494671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 294/572] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:10.495436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad1af3-2252-433f-9baf-b253d513a1eb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.545674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:10.586104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f954886-6679-46b0-96ec-f05b81f2ee42 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.594928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad1af3-2252-433f-9baf-b253d513a1eb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.619717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d46763-942f-4db2-97f4-9c93b9778340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.620502 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d46763-942f-4db2-97f4-9c93b9778340 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.620785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d46763-942f-4db2-97f4-9c93b9778340 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.621061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d46763-942f-4db2-97f4-9c93b9778340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.629271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acb366-5c0f-47eb-992a-7699c0fee93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.630483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acb366-5c0f-47eb-992a-7699c0fee93f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.630762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acb366-5c0f-47eb-992a-7699c0fee93f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.631028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acb366-5c0f-47eb-992a-7699c0fee93f None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.631862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acb366-5c0f-47eb-992a-7699c0fee93f None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:10.632162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acb366-5c0f-47eb-992a-7699c0fee93f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:10.632333 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d46763-942f-4db2-97f4-9c93b9778340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:10.633562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acb366-5c0f-47eb-992a-7699c0fee93f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:10.633847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d46763-942f-4db2-97f4-9c93b9778340 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:10.641368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 295/573] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:10.642151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acb366-5c0f-47eb-992a-7699c0fee93f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-69d46763-942f-4db2-97f4-9c93b9778340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:10.655348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:10.656337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:10.657269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:10.657269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:10.657269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:10.657269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:10.657269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 279/574] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:10.657714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d46763-942f-4db2-97f4-9c93b9778340 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.695266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:10.741481 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acb366-5c0f-47eb-992a-7699c0fee93f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.756450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d46763-942f-4db2-97f4-9c93b9778340 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.766619 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:10.776850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89217-a0f9-4bc1-a54d-bf38e60e158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.777560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89217-a0f9-4bc1-a54d-bf38e60e158a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.777842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89217-a0f9-4bc1-a54d-bf38e60e158a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.778894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89217-a0f9-4bc1-a54d-bf38e60e158a None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.778894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89217-a0f9-4bc1-a54d-bf38e60e158a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:10.779183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89217-a0f9-4bc1-a54d-bf38e60e158a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:10.780605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89217-a0f9-4bc1-a54d-bf38e60e158a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:10.790441 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89e34d-1eb4-4662-9261-ad42eefeeeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.791232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89e34d-1eb4-4662-9261-ad42eefeeeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.791518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89e34d-1eb4-4662-9261-ad42eefeeeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.791794 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89e34d-1eb4-4662-9261-ad42eefeeeb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.792708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 296/575] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:10.793370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89217-a0f9-4bc1-a54d-bf38e60e158a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.804830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89e34d-1eb4-4662-9261-ad42eefeeeb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:10.806354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89e34d-1eb4-4662-9261-ad42eefeeeb3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-0c89e34d-1eb4-4662-9261-ad42eefeeeb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:10.821573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:10.822493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:10.823610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:10.823610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:10.823610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:10.823610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:10.823610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:10.823610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:10.823610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:10.823610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:10.823610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 280/576] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:10.823610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89e34d-1eb4-4662-9261-ad42eefeeeb3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.900391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89217-a0f9-4bc1-a54d-bf38e60e158a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.913550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:10.921979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89e34d-1eb4-4662-9261-ad42eefeeeb3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:10.934468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12d80-cf8e-4889-ba92-67ec40aae956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.935290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12d80-cf8e-4889-ba92-67ec40aae956 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.935764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12d80-cf8e-4889-ba92-67ec40aae956 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.936046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12d80-cf8e-4889-ba92-67ec40aae956 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.946787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d12d80-cf8e-4889-ba92-67ec40aae956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-33d12d80-cf8e-4889-ba92-67ec40aae956 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:10.954804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:10.955795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:10.956635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 297/577] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:10.957524 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49316f67-c5b6-4fb2-b03a-4b58e6f539c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:10.957625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12d80-cf8e-4889-ba92-67ec40aae956 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:10.957721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49316f67-c5b6-4fb2-b03a-4b58e6f539c0 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:10.957721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49316f67-c5b6-4fb2-b03a-4b58e6f539c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:10.957721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49316f67-c5b6-4fb2-b03a-4b58e6f539c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:10.970839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49316f67-c5b6-4fb2-b03a-4b58e6f539c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:10.972322 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49316f67-c5b6-4fb2-b03a-4b58e6f539c0 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-49316f67-c5b6-4fb2-b03a-4b58e6f539c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:10.990748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:10.991694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:10.992773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:10.992773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:10.992773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:10.992773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:10.992773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 281/578] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:10.992773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49316f67-c5b6-4fb2-b03a-4b58e6f539c0 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:11.017209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:11.075867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12d80-cf8e-4889-ba92-67ec40aae956 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:11.097328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49316f67-c5b6-4fb2-b03a-4b58e6f539c0 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:11.109329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eec160-fabf-4cfc-abe3-5c25f55a6e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:11.109999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eec160-fabf-4cfc-abe3-5c25f55a6e70 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:11.110284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eec160-fabf-4cfc-abe3-5c25f55a6e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:11.110563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eec160-fabf-4cfc-abe3-5c25f55a6e70 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:11.111283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eec160-fabf-4cfc-abe3-5c25f55a6e70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:11.111591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eec160-fabf-4cfc-abe3-5c25f55a6e70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:11.112960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eec160-fabf-4cfc-abe3-5c25f55a6e70 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:11.126056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 298/579] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:11.126627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eec160-fabf-4cfc-abe3-5c25f55a6e70 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:11.132556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138963c4-7c17-4af4-9d27-ba120d22cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:11.133230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138963c4-7c17-4af4-9d27-ba120d22cea8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:11.133517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138963c4-7c17-4af4-9d27-ba120d22cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:11.133792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138963c4-7c17-4af4-9d27-ba120d22cea8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:11.134626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138963c4-7c17-4af4-9d27-ba120d22cea8 None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=42a5ec793f014cefb9d0a0d7258bbb59) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:11.134938 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138963c4-7c17-4af4-9d27-ba120d22cea8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:11.136386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138963c4-7c17-4af4-9d27-ba120d22cea8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:11.141874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 282/580] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:10 2022] GET /identity/v3/role_assignments?user.id=42a5ec793f014cefb9d0a0d7258bbb59&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:11.142647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138963c4-7c17-4af4-9d27-ba120d22cea8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:11.159493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:11.240893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eec160-fabf-4cfc-abe3-5c25f55a6e70 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:11.245402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:11.261805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138963c4-7c17-4af4-9d27-ba120d22cea8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:11.283101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc856a-0eed-411b-8d4f-fb2590808f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:11.284447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc856a-0eed-411b-8d4f-fb2590808f4b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:11.284447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc856a-0eed-411b-8d4f-fb2590808f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:11.284648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc856a-0eed-411b-8d4f-fb2590808f4b None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:11.285517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc856a-0eed-411b-8d4f-fb2590808f4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:11.285806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc856a-0eed-411b-8d4f-fb2590808f4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:11.287742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc856a-0eed-411b-8d4f-fb2590808f4b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:11.299317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc90e1a-5f93-4477-9b93-71327691acf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:11.300270 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 299/581] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:11.300516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc90e1a-5f93-4477-9b93-71327691acf7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:11.301084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc90e1a-5f93-4477-9b93-71327691acf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:11.301431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc856a-0eed-411b-8d4f-fb2590808f4b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:11.302631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc90e1a-5f93-4477-9b93-71327691acf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:11.303621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc90e1a-5f93-4477-9b93-71327691acf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:11.304293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc90e1a-5f93-4477-9b93-71327691acf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:11.306057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc90e1a-5f93-4477-9b93-71327691acf7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:11.315025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 283/582] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:11.316140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc90e1a-5f93-4477-9b93-71327691acf7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:11.401860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:11.414894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc856a-0eed-411b-8d4f-fb2590808f4b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:11.435731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc90e1a-5f93-4477-9b93-71327691acf7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:11.444832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35c124-21f1-45a7-8db4-b39efe008ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:11.447075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35c124-21f1-45a7-8db4-b39efe008ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:11.448263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35c124-21f1-45a7-8db4-b39efe008ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:11.448540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35c124-21f1-45a7-8db4-b39efe008ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:11.449294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35c124-21f1-45a7-8db4-b39efe008ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:11.449599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35c124-21f1-45a7-8db4-b39efe008ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:11.451208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35c124-21f1-45a7-8db4-b39efe008ad8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:11.469122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 300/583] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:11.469122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35c124-21f1-45a7-8db4-b39efe008ad8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:11.495458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a7be6-9c4c-4c66-84fd-eb2491d984ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:11.496439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a7be6-9c4c-4c66-84fd-eb2491d984ed None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:11.496724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a7be6-9c4c-4c66-84fd-eb2491d984ed None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:11.497009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a7be6-9c4c-4c66-84fd-eb2491d984ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:11.505272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a7be6-9c4c-4c66-84fd-eb2491d984ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:11.506726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a7be6-9c4c-4c66-84fd-eb2491d984ed None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-083a7be6-9c4c-4c66-84fd-eb2491d984ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:11.521621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:11.522591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 284/584] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jul 14 11:24:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:11.523627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a7be6-9c4c-4c66-84fd-eb2491d984ed None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:11.536841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:11.587702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35c124-21f1-45a7-8db4-b39efe008ad8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:11.624004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce309e-228f-4c9f-b90a-8c44150d6cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:11.624892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce309e-228f-4c9f-b90a-8c44150d6cd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:11.625189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce309e-228f-4c9f-b90a-8c44150d6cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:11.625471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce309e-228f-4c9f-b90a-8c44150d6cd4 None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/a1a7213cbf6448b7ba6570a52a4c7c0e/roles/60130d2ba383400091dc5f253d40a43a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:11.631998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a7be6-9c4c-4c66-84fd-eb2491d984ed None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:11.640738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce309e-228f-4c9f-b90a-8c44150d6cd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=a1a7213cbf6448b7ba6570a52a4c7c0e, role_id=60130d2ba383400091dc5f253d40a43a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:11.642212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce309e-228f-4c9f-b90a-8c44150d6cd4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:11.667035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d12f02-826b-4b44-9255-ee49bb7760ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:11.667748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d12f02-826b-4b44-9255-ee49bb7760ed None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:11.668031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d12f02-826b-4b44-9255-ee49bb7760ed None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:11.668306 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d12f02-826b-4b44-9255-ee49bb7760ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:11.679381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d12f02-826b-4b44-9255-ee49bb7760ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:11.680819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d12f02-826b-4b44-9255-ee49bb7760ed None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-f2d12f02-826b-4b44-9255-ee49bb7760ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:11.698354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:11.701757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:11.702867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:11.702867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:11.702867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:11.702867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:11.702867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 285/585] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:11.702867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d12f02-826b-4b44-9255-ee49bb7760ed None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:11.733774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 301/586] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:24:11 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/a1a7213cbf6448b7ba6570a52a4c7c0e/roles/60130d2ba383400091dc5f253d40a43a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:11.746365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce309e-228f-4c9f-b90a-8c44150d6cd4 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:11.749615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:11.776865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:11.828399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d12f02-826b-4b44-9255-ee49bb7760ed None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:11.878678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6c751-4b84-4abb-b9a1-849adb0b413e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:11.879926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6c751-4b84-4abb-b9a1-849adb0b413e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:11.880418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6c751-4b84-4abb-b9a1-849adb0b413e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:11.880896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6c751-4b84-4abb-b9a1-849adb0b413e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:11.890272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6c751-4b84-4abb-b9a1-849adb0b413e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:11.892129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6c751-4b84-4abb-b9a1-849adb0b413e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:11.903032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce309e-228f-4c9f-b90a-8c44150d6cd4 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-80f6c751-4b84-4abb-b9a1-849adb0b413e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:11.909633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:11.913796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:11.914732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 286/587] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:11.916211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6c751-4b84-4abb-b9a1-849adb0b413e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:11.950223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314286d-464b-4a7e-aea0-052f8a8e17e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:11.951402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314286d-464b-4a7e-aea0-052f8a8e17e7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:11.951961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314286d-464b-4a7e-aea0-052f8a8e17e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:11.952240 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314286d-464b-4a7e-aea0-052f8a8e17e7 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:11.953286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314286d-464b-4a7e-aea0-052f8a8e17e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:11.953912 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314286d-464b-4a7e-aea0-052f8a8e17e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:11.955689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314286d-464b-4a7e-aea0-052f8a8e17e7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:11.979137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 302/588] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jul 14 11:24:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:11.979990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314286d-464b-4a7e-aea0-052f8a8e17e7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:12.048923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6c751-4b84-4abb-b9a1-849adb0b413e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:12.050160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:12.073649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:12.089747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e25a17-972c-4501-9e8b-ff7a43199937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:12.090545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e25a17-972c-4501-9e8b-ff7a43199937 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:12.090832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e25a17-972c-4501-9e8b-ff7a43199937 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:12.091114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e25a17-972c-4501-9e8b-ff7a43199937 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:12.091991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e25a17-972c-4501-9e8b-ff7a43199937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:12.092303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e25a17-972c-4501-9e8b-ff7a43199937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:12.093726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e25a17-972c-4501-9e8b-ff7a43199937 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:12.106655 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314286d-464b-4a7e-aea0-052f8a8e17e7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:12.107559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 287/589] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:12.108576 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e25a17-972c-4501-9e8b-ff7a43199937 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:12.153293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de137-1e53-426d-8089-b44a778b22b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:12.155127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de137-1e53-426d-8089-b44a778b22b9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:12.155647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de137-1e53-426d-8089-b44a778b22b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:12.156118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de137-1e53-426d-8089-b44a778b22b9 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:12.157678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6de137-1e53-426d-8089-b44a778b22b9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:12.158616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6de137-1e53-426d-8089-b44a778b22b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:12.160594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6de137-1e53-426d-8089-b44a778b22b9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:12.189440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 303/590] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jul 14 11:24:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:12.190927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de137-1e53-426d-8089-b44a778b22b9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:12.238544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e25a17-972c-4501-9e8b-ff7a43199937 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:12.280782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62090d-a5ea-4d15-a84e-cb0a9e63eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:12.281668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62090d-a5ea-4d15-a84e-cb0a9e63eff8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:12.281962 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62090d-a5ea-4d15-a84e-cb0a9e63eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:12.282231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62090d-a5ea-4d15-a84e-cb0a9e63eff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:12.293263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62090d-a5ea-4d15-a84e-cb0a9e63eff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:12.294718 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62090d-a5ea-4d15-a84e-cb0a9e63eff8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:12.311744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de137-1e53-426d-8089-b44a778b22b9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-3c62090d-a5ea-4d15-a84e-cb0a9e63eff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:12.317117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:12.318020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:12.318906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:12.318906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:12.318906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:12.318906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:12.321284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 288/591] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:12.322310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62090d-a5ea-4d15-a84e-cb0a9e63eff8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:12.337149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:12.358863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17099dc-b362-4432-a5ba-708b2d376218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:12.360171 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17099dc-b362-4432-a5ba-708b2d376218 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:12.360786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17099dc-b362-4432-a5ba-708b2d376218 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:12.361269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17099dc-b362-4432-a5ba-708b2d376218 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:12.361640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:12.370594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17099dc-b362-4432-a5ba-708b2d376218 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:12.372754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17099dc-b362-4432-a5ba-708b2d376218 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-e17099dc-b362-4432-a5ba-708b2d376218 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:12.392266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:12.393286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:12.394206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:12.394206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:12.394206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:12.394206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:12.394206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:12.394206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:12.394206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:12.394206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:12.394206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 11:24:12.394206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:12.397802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 304/592] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:12.398918 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17099dc-b362-4432-a5ba-708b2d376218 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:12.446973 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62090d-a5ea-4d15-a84e-cb0a9e63eff8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:12.491331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fae85-0b31-4da3-ad58-2f6c7aed3ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:12.492827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fae85-0b31-4da3-ad58-2f6c7aed3ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:12.493317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fae85-0b31-4da3-ad58-2f6c7aed3ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:12.494110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fae85-0b31-4da3-ad58-2f6c7aed3ed4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:12.510302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fae85-0b31-4da3-ad58-2f6c7aed3ed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:12.511938 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fae85-0b31-4da3-ad58-2f6c7aed3ed4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-918fae85-0b31-4da3-ad58-2f6c7aed3ed4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:12.532274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:12.533426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:12.534510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:12.534510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:12.534510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:12.534510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:12.534510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 289/593] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:12.535278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fae85-0b31-4da3-ad58-2f6c7aed3ed4 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:12.535459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17099dc-b362-4432-a5ba-708b2d376218 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:12.571142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091be686-f460-4be4-93cf-3a2c0342fb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:12.572198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091be686-f460-4be4-93cf-3a2c0342fb37 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:12.572677 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091be686-f460-4be4-93cf-3a2c0342fb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:12.573146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091be686-f460-4be4-93cf-3a2c0342fb37 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:12.574088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091be686-f460-4be4-93cf-3a2c0342fb37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:12.574586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091be686-f460-4be4-93cf-3a2c0342fb37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:12.576405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091be686-f460-4be4-93cf-3a2c0342fb37 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:12.588644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 305/594] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:12.589891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091be686-f460-4be4-93cf-3a2c0342fb37 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:12.641673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:12.642121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:12.662411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fae85-0b31-4da3-ad58-2f6c7aed3ed4 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:12.705070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18436d89-e3fb-4d8e-b841-2b81bfb248fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:12.706508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18436d89-e3fb-4d8e-b841-2b81bfb248fe None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:12.707073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18436d89-e3fb-4d8e-b841-2b81bfb248fe None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:12.707522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18436d89-e3fb-4d8e-b841-2b81bfb248fe None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:12.709302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18436d89-e3fb-4d8e-b841-2b81bfb248fe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:12.710079 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18436d89-e3fb-4d8e-b841-2b81bfb248fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:12.712097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18436d89-e3fb-4d8e-b841-2b81bfb248fe None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:12.719597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091be686-f460-4be4-93cf-3a2c0342fb37 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:12.734395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 290/595] 10.222.0.40 () {56 vars in 1139 bytes} [Thu Jul 14 11:24:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:12.735492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18436d89-e3fb-4d8e-b841-2b81bfb248fe None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:12.768679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39c260-a93b-4067-8ada-02bcce7ad847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:12.770100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39c260-a93b-4067-8ada-02bcce7ad847 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:12.771227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39c260-a93b-4067-8ada-02bcce7ad847 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:12.771574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39c260-a93b-4067-8ada-02bcce7ad847 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:12.772621 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39c260-a93b-4067-8ada-02bcce7ad847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:12.773004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39c260-a93b-4067-8ada-02bcce7ad847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:12.774904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39c260-a93b-4067-8ada-02bcce7ad847 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:12.790966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 306/596] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:12.791962 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39c260-a93b-4067-8ada-02bcce7ad847 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:12.864912 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18436d89-e3fb-4d8e-b841-2b81bfb248fe None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:12.895267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:12.899663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39c260-a93b-4067-8ada-02bcce7ad847 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:12.907430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bd327-adc8-4585-95f1-7f00fd24c1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:12.908517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bd327-adc8-4585-95f1-7f00fd24c1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:12.908899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bd327-adc8-4585-95f1-7f00fd24c1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:12.909349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bd327-adc8-4585-95f1-7f00fd24c1d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:12.916932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bd327-adc8-4585-95f1-7f00fd24c1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:12.918691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bd327-adc8-4585-95f1-7f00fd24c1d0 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-a79bd327-adc8-4585-95f1-7f00fd24c1d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:12.934444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:12.935559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:12.936447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:12.936447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:12.936447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:12.936447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:12.936447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:12.936447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:12.936447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:12.936447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:12.940963 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 291/597] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:12.943180 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bd327-adc8-4585-95f1-7f00fd24c1d0 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:12.947574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bac346-f98b-4c9e-9941-66102af5c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:12.947744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:12.948730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bac346-f98b-4c9e-9941-66102af5c07d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:12.949250 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bac346-f98b-4c9e-9941-66102af5c07d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:12.949738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bac346-f98b-4c9e-9941-66102af5c07d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:12.961861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bac346-f98b-4c9e-9941-66102af5c07d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:12.963628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bac346-f98b-4c9e-9941-66102af5c07d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-62bac346-f98b-4c9e-9941-66102af5c07d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:12.982174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:12.983196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:12.984060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:12.984060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:12.984060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:12.984060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:12.986060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 307/598] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:12.987200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bac346-f98b-4c9e-9941-66102af5c07d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:13.065062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bd327-adc8-4585-95f1-7f00fd24c1d0 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:13.099831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bac346-f98b-4c9e-9941-66102af5c07d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:13.108192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b4d21-4d4c-4c93-9677-707603556acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:13.109039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b4d21-4d4c-4c93-9677-707603556acb None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.109349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b4d21-4d4c-4c93-9677-707603556acb None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.109630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b4d21-4d4c-4c93-9677-707603556acb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.117724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b4d21-4d4c-4c93-9677-707603556acb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:13.119565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b4d21-4d4c-4c93-9677-707603556acb None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-7a3b4d21-4d4c-4c93-9677-707603556acb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:13.136448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:13.137409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:13.138924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:13.138924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:13.138924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:13.138924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:13.138924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:13.138924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:13.138924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:13.138924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:13.138924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 292/599] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:13.138924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b4d21-4d4c-4c93-9677-707603556acb None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:13.140100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21d74f-525b-4f60-a97f-cbc47a386ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:13.141344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21d74f-525b-4f60-a97f-cbc47a386ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.141833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21d74f-525b-4f60-a97f-cbc47a386ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.142598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21d74f-525b-4f60-a97f-cbc47a386ff8 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.143593 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21d74f-525b-4f60-a97f-cbc47a386ff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:13.144095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21d74f-525b-4f60-a97f-cbc47a386ff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:13.145828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21d74f-525b-4f60-a97f-cbc47a386ff8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:13.158633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 308/600] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:13.159865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21d74f-525b-4f60-a97f-cbc47a386ff8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:13.177370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:13.214444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:13.260569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b4d21-4d4c-4c93-9677-707603556acb None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:13.278218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21d74f-525b-4f60-a97f-cbc47a386ff8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:13.300515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36f574-15e3-4ae7-9e4f-bb53d975395e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:13.301369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36f574-15e3-4ae7-9e4f-bb53d975395e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.302337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36f574-15e3-4ae7-9e4f-bb53d975395e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.302618 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36f574-15e3-4ae7-9e4f-bb53d975395e None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.303704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36f574-15e3-4ae7-9e4f-bb53d975395e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:13.304456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36f574-15e3-4ae7-9e4f-bb53d975395e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:13.306315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36f574-15e3-4ae7-9e4f-bb53d975395e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:13.314397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508eb1f4-1f4d-4ed2-a091-e206a159530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:13.315055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508eb1f4-1f4d-4ed2-a091-e206a159530c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.315390 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508eb1f4-1f4d-4ed2-a091-e206a159530c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.315612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508eb1f4-1f4d-4ed2-a091-e206a159530c None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.316401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508eb1f4-1f4d-4ed2-a091-e206a159530c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:13.316710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508eb1f4-1f4d-4ed2-a091-e206a159530c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:13.318263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508eb1f4-1f4d-4ed2-a091-e206a159530c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:13.321302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 293/601] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:13.322241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36f574-15e3-4ae7-9e4f-bb53d975395e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:13.340498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 309/602] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:13.342197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb457053-cea5-4b8f-a53d-0d4a63bff216 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.342673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb457053-cea5-4b8f-a53d-0d4a63bff216 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.343129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb457053-cea5-4b8f-a53d-0d4a63bff216 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.344593 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 310/603] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:13.345499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb457053-cea5-4b8f-a53d-0d4a63bff216 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:13.425139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:13.434858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36f574-15e3-4ae7-9e4f-bb53d975395e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:13.460028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb457053-cea5-4b8f-a53d-0d4a63bff216 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:13.460348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:13.475800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94171d1-779a-41e7-b7d1-4e324c1626f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:13.477353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94171d1-779a-41e7-b7d1-4e324c1626f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.477353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94171d1-779a-41e7-b7d1-4e324c1626f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.477353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94171d1-779a-41e7-b7d1-4e324c1626f6 None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/147d764ef17042dbb7ea179c29711588/roles/60130d2ba383400091dc5f253d40a43a` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.495263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94171d1-779a-41e7-b7d1-4e324c1626f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=147d764ef17042dbb7ea179c29711588, role_id=60130d2ba383400091dc5f253d40a43a)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:13.496798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94171d1-779a-41e7-b7d1-4e324c1626f6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:13.509296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e3efd-80b4-44ca-a582-d28f8d0e983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:13.517553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e3efd-80b4-44ca-a582-d28f8d0e983c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.517553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e3efd-80b4-44ca-a582-d28f8d0e983c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.517553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e3efd-80b4-44ca-a582-d28f8d0e983c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.517857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e3efd-80b4-44ca-a582-d28f8d0e983c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:13.519001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e3efd-80b4-44ca-a582-d28f8d0e983c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4c7e3efd-80b4-44ca-a582-d28f8d0e983c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:13.532797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:13.533937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:13.534834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:13.534834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:13.534834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:13.534834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:13.534834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:13.534834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:13.534834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:13.534834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:13.534834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 311/604] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:13.534834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e3efd-80b4-44ca-a582-d28f8d0e983c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:13.593753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 294/605] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:24:13 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/147d764ef17042dbb7ea179c29711588/roles/60130d2ba383400091dc5f253d40a43a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:13.595389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd0745-89da-49cb-a085-466ae1b27f3a None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.595861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd0745-89da-49cb-a085-466ae1b27f3a None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.596140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd0745-89da-49cb-a085-466ae1b27f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.624260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-90bd0745-89da-49cb-a085-466ae1b27f3a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:13.624796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-90bd0745-89da-49cb-a085-466ae1b27f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:13.670155 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e3efd-80b4-44ca-a582-d28f8d0e983c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:13.709183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 295/606] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:13.709183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd0745-89da-49cb-a085-466ae1b27f3a None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:13.731581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:13.739947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79a397-7fe2-426b-903b-2b79ff3db805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:13.741496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79a397-7fe2-426b-903b-2b79ff3db805 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.741957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79a397-7fe2-426b-903b-2b79ff3db805 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.746446 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79a397-7fe2-426b-903b-2b79ff3db805 None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/f12e348368ac42c696623b4a8f781977/roles/60130d2ba383400091dc5f253d40a43a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.762085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79a397-7fe2-426b-903b-2b79ff3db805 None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=f12e348368ac42c696623b4a8f781977, role_id=60130d2ba383400091dc5f253d40a43a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:13.764734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79a397-7fe2-426b-903b-2b79ff3db805 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:13.837065 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd0745-89da-49cb-a085-466ae1b27f3a None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:13.839604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:13.867739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 312/607] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:24:13 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/f12e348368ac42c696623b4a8f781977/roles/60130d2ba383400091dc5f253d40a43a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:13.867739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79a397-7fe2-426b-903b-2b79ff3db805 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:13.900549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed25ff-1d9e-4af3-9ae7-5b50740de4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:13.902188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed25ff-1d9e-4af3-9ae7-5b50740de4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.902188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed25ff-1d9e-4af3-9ae7-5b50740de4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.902188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed25ff-1d9e-4af3-9ae7-5b50740de4d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.902789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed25ff-1d9e-4af3-9ae7-5b50740de4d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:13.903100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed25ff-1d9e-4af3-9ae7-5b50740de4d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:13.904640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed25ff-1d9e-4af3-9ae7-5b50740de4d0 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:13.919040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 296/608] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:13.920524 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97690e26-661e-4de9-a808-6c1a42cf3a21 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:13.920817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97690e26-661e-4de9-a808-6c1a42cf3a21 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:13.921220 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97690e26-661e-4de9-a808-6c1a42cf3a21 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:13.926785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 297/609] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:13.927739 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97690e26-661e-4de9-a808-6c1a42cf3a21 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:14.015534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79a397-7fe2-426b-903b-2b79ff3db805 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:14.046195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97690e26-661e-4de9-a808-6c1a42cf3a21 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:14.065187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70310-0e9b-4870-a692-db071a00bf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:14.065187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70310-0e9b-4870-a692-db071a00bf37 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:14.065187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70310-0e9b-4870-a692-db071a00bf37 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:14.065187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70310-0e9b-4870-a692-db071a00bf37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:14.080775 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af70310-0e9b-4870-a692-db071a00bf37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:14.088299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f091ac-8643-4437-9a82-da46c0fc77d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:14.089332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f091ac-8643-4437-9a82-da46c0fc77d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:14.089630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f091ac-8643-4437-9a82-da46c0fc77d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:14.089921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f091ac-8643-4437-9a82-da46c0fc77d3 None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/481ef56b1cfa465a8efc59736a34e412/roles/60130d2ba383400091dc5f253d40a43a` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-8af70310-0e9b-4870-a692-db071a00bf37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:14.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:14.091672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:14.093115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 313/610] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:14.095144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc3e8f-1641-4eb1-b450-dc991f6d48e4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:14.095144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc3e8f-1641-4eb1-b450-dc991f6d48e4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:14.095338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc3e8f-1641-4eb1-b450-dc991f6d48e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:14.106645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f091ac-8643-4437-9a82-da46c0fc77d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=481ef56b1cfa465a8efc59736a34e412, role_id=60130d2ba383400091dc5f253d40a43a)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:14.108356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f091ac-8643-4437-9a82-da46c0fc77d3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:14.115664 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-60cc3e8f-1641-4eb1-b450-dc991f6d48e4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:14.116860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-60cc3e8f-1641-4eb1-b450-dc991f6d48e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:14.153834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:14.193197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 314/611] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:14.193197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc3e8f-1641-4eb1-b450-dc991f6d48e4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:14.220367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 298/612] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:24:13 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/481ef56b1cfa465a8efc59736a34e412/roles/60130d2ba383400091dc5f253d40a43a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:14.234288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f091ac-8643-4437-9a82-da46c0fc77d3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:14.257553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:14.373817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc3e8f-1641-4eb1-b450-dc991f6d48e4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:14.426956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995e9ad-f109-415c-a851-af8d4820f416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:14.427572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995e9ad-f109-415c-a851-af8d4820f416 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:14.427847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995e9ad-f109-415c-a851-af8d4820f416 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:14.428115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995e9ad-f109-415c-a851-af8d4820f416 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:14.428871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995e9ad-f109-415c-a851-af8d4820f416 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:14.430507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995e9ad-f109-415c-a851-af8d4820f416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:14.432371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995e9ad-f109-415c-a851-af8d4820f416 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:14.442980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f091ac-8643-4437-9a82-da46c0fc77d3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:14.446587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 315/613] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:14.449893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995e9ad-f109-415c-a851-af8d4820f416 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:14.483959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd426-9bc8-448e-a083-fab12e54dfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:14.484860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd426-9bc8-448e-a083-fab12e54dfed None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:14.485134 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd426-9bc8-448e-a083-fab12e54dfed None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:14.485458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd426-9bc8-448e-a083-fab12e54dfed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:14.489601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:14.501714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fd426-9bc8-448e-a083-fab12e54dfed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-e00fd426-9bc8-448e-a083-fab12e54dfed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:14.513246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:14.514171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:14.515276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:14.520199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 299/614] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:14.526576 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd426-9bc8-448e-a083-fab12e54dfed None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:14.592070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995e9ad-f109-415c-a851-af8d4820f416 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:14.637265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a06e21-2273-4fcb-a3df-fed3822d82a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:14.638307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a06e21-2273-4fcb-a3df-fed3822d82a6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:14.638731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a06e21-2273-4fcb-a3df-fed3822d82a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:14.639142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a06e21-2273-4fcb-a3df-fed3822d82a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:14.648443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:14.665964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a06e21-2273-4fcb-a3df-fed3822d82a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:14.667673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a06e21-2273-4fcb-a3df-fed3822d82a6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:14.687650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd426-9bc8-448e-a083-fab12e54dfed None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-a9a06e21-2273-4fcb-a3df-fed3822d82a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:14.692840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:14.693755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:14.694634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:14.694634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:14.694634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:14.694634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:14.701273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 316/615] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:14.703023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a06e21-2273-4fcb-a3df-fed3822d82a6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:14.758147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5f6cd-ea58-4503-90f0-3ad40c0b5c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:14.759746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f6cd-ea58-4503-90f0-3ad40c0b5c09 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:14.761125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f6cd-ea58-4503-90f0-3ad40c0b5c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:14.762733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f6cd-ea58-4503-90f0-3ad40c0b5c09 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:14.763706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5f6cd-ea58-4503-90f0-3ad40c0b5c09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:14.765604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5f6cd-ea58-4503-90f0-3ad40c0b5c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:14.767211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5f6cd-ea58-4503-90f0-3ad40c0b5c09 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:14.785567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 300/616] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:14.791970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5f6cd-ea58-4503-90f0-3ad40c0b5c09 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:14.885129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:14.904858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a06e21-2273-4fcb-a3df-fed3822d82a6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:14.974003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f52ffb-1239-4038-8023-5e91999cb12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:14.974588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f52ffb-1239-4038-8023-5e91999cb12a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:14.976033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f52ffb-1239-4038-8023-5e91999cb12a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:14.976319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f52ffb-1239-4038-8023-5e91999cb12a None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:14.977047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f52ffb-1239-4038-8023-5e91999cb12a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:14.977378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f52ffb-1239-4038-8023-5e91999cb12a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:14.979001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f52ffb-1239-4038-8023-5e91999cb12a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:15.001408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 317/617] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:15.005855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5f6cd-ea58-4503-90f0-3ad40c0b5c09 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:15.008810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f52ffb-1239-4038-8023-5e91999cb12a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:15.067010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f77af-ad3a-4ab3-8499-a9c9948fb912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:15.068932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f77af-ad3a-4ab3-8499-a9c9948fb912 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:15.069976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f77af-ad3a-4ab3-8499-a9c9948fb912 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:15.070611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f77af-ad3a-4ab3-8499-a9c9948fb912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:15.074695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:15.093180 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f77af-ad3a-4ab3-8499-a9c9948fb912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:15.095863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f77af-ad3a-4ab3-8499-a9c9948fb912 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-157f77af-ad3a-4ab3-8499-a9c9948fb912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:15.120593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:15.121812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:15.122728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:15.122728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:15.122728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:15.122728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:15.124849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 301/618] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:15.130003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f77af-ad3a-4ab3-8499-a9c9948fb912 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:15.177584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f52ffb-1239-4038-8023-5e91999cb12a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:15.212033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd35957-347c-4de6-b481-5a9df78a7422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:15.212824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35957-347c-4de6-b481-5a9df78a7422 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:15.213099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35957-347c-4de6-b481-5a9df78a7422 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:15.213466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35957-347c-4de6-b481-5a9df78a7422 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:15.222184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd35957-347c-4de6-b481-5a9df78a7422 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:15.224351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd35957-347c-4de6-b481-5a9df78a7422 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-afd35957-347c-4de6-b481-5a9df78a7422 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:15.240940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:15.242097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:15.243046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:15.243046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:15.243046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:15.243046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:15.243046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:15.243046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:15.243046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:15.243046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:15.243046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:15.243046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:15.246927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 318/619] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:15.248191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd35957-347c-4de6-b481-5a9df78a7422 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:15.253451 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:15.328266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f77af-ad3a-4ab3-8499-a9c9948fb912 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:15.357208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd35957-347c-4de6-b481-5a9df78a7422 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:15.391718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd8db3-5c1f-4dc0-99f0-33204ecb1b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:15.392411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd8db3-5c1f-4dc0-99f0-33204ecb1b65 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:15.392674 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd8db3-5c1f-4dc0-99f0-33204ecb1b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:15.392947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd8db3-5c1f-4dc0-99f0-33204ecb1b65 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:15.393732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3b4f8-31a4-4728-b2ab-d8e7b5981d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:15.393872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd8db3-5c1f-4dc0-99f0-33204ecb1b65 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:15.394114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd8db3-5c1f-4dc0-99f0-33204ecb1b65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:15.394930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3b4f8-31a4-4728-b2ab-d8e7b5981d63 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:15.395986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3b4f8-31a4-4728-b2ab-d8e7b5981d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:15.396358 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd8db3-5c1f-4dc0-99f0-33204ecb1b65 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:15.396695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3b4f8-31a4-4728-b2ab-d8e7b5981d63 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:15.397698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3b4f8-31a4-4728-b2ab-d8e7b5981d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:15.398317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3b4f8-31a4-4728-b2ab-d8e7b5981d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:15.399992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3b4f8-31a4-4728-b2ab-d8e7b5981d63 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:15.413946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 302/620] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:15.419702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3b4f8-31a4-4728-b2ab-d8e7b5981d63 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:15.420717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 319/621] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jul 14 11:24:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:15.421955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd8db3-5c1f-4dc0-99f0-33204ecb1b65 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:15.486516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:15.487912 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:15.528144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd8db3-5c1f-4dc0-99f0-33204ecb1b65 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:15.565415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3b4f8-31a4-4728-b2ab-d8e7b5981d63 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:15.568142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b201b83-9ab3-4a5c-bbca-f6daa003cd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:15.569384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b201b83-9ab3-4a5c-bbca-f6daa003cd98 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:15.570502 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b201b83-9ab3-4a5c-bbca-f6daa003cd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:15.570901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b201b83-9ab3-4a5c-bbca-f6daa003cd98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:15.591615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b201b83-9ab3-4a5c-bbca-f6daa003cd98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:15.594743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b201b83-9ab3-4a5c-bbca-f6daa003cd98 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-6b201b83-9ab3-4a5c-bbca-f6daa003cd98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:15.621395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:15.622489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:15.623571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:15.623571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:15.623571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:15.623571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:15.637448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 320/622] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:15.639999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b201b83-9ab3-4a5c-bbca-f6daa003cd98 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:15.646178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cea52-796e-478f-a344-45b3d9561914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:15.646976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cea52-796e-478f-a344-45b3d9561914 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:15.647254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cea52-796e-478f-a344-45b3d9561914 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:15.647523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cea52-796e-478f-a344-45b3d9561914 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:15.662619 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cea52-796e-478f-a344-45b3d9561914 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:15.664070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cea52-796e-478f-a344-45b3d9561914 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-d53cea52-796e-478f-a344-45b3d9561914 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:15.690537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:15.691538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:15.693009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:15.693009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:15.693009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:15.693009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:15.693009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:15.693009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:15.693009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:15.693009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:15.693009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 11:24:15.693009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:15.694777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 303/623] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:15.700455 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cea52-796e-478f-a344-45b3d9561914 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:15.847102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cea52-796e-478f-a344-45b3d9561914 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:15.848610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b201b83-9ab3-4a5c-bbca-f6daa003cd98 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:15.857543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:15.880734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:15.917287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee11b0-741d-45e1-95e2-511ba26a6abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:15.917975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee11b0-741d-45e1-95e2-511ba26a6abe None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:15.918267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee11b0-741d-45e1-95e2-511ba26a6abe None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:15.918553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee11b0-741d-45e1-95e2-511ba26a6abe None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:15.919415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee11b0-741d-45e1-95e2-511ba26a6abe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:15.919737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee11b0-741d-45e1-95e2-511ba26a6abe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:15.921781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee11b0-741d-45e1-95e2-511ba26a6abe None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:15.925179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638b7b3-9040-4776-8545-7feeaf231f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:15.925886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638b7b3-9040-4776-8545-7feeaf231f70 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:15.943454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638b7b3-9040-4776-8545-7feeaf231f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:15.943454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638b7b3-9040-4776-8545-7feeaf231f70 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:15.943454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638b7b3-9040-4776-8545-7feeaf231f70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:15.943454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638b7b3-9040-4776-8545-7feeaf231f70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:15.943454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638b7b3-9040-4776-8545-7feeaf231f70 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:15.954748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 321/624] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:15.955906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 304/625] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jul 14 11:24:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:15.956835 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480539e8-03bc-4b72-98ce-ca7e286c4b0f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:15.957128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480539e8-03bc-4b72-98ce-ca7e286c4b0f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:15.957465 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480539e8-03bc-4b72-98ce-ca7e286c4b0f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:15.958913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 322/626] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:15.960766 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480539e8-03bc-4b72-98ce-ca7e286c4b0f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:15.961693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee11b0-741d-45e1-95e2-511ba26a6abe None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:16.067748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee11b0-741d-45e1-95e2-511ba26a6abe None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:16.090604 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480539e8-03bc-4b72-98ce-ca7e286c4b0f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:16.104100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb008c-b9bd-4bd8-b281-ee29ffae2ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:16.104896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb008c-b9bd-4bd8-b281-ee29ffae2ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.105193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb008c-b9bd-4bd8-b281-ee29ffae2ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.105475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb008c-b9bd-4bd8-b281-ee29ffae2ba9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.113041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb008c-b9bd-4bd8-b281-ee29ffae2ba9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:16.115254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb008c-b9bd-4bd8-b281-ee29ffae2ba9 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-cbdb008c-b9bd-4bd8-b281-ee29ffae2ba9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:16.130374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:16.131343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:16.132263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:16.132263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:16.132263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:16.132263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:16.132263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:16.132263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:16.132263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:16.132263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:16.132263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 305/627] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:16.133262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddde39-48ed-447c-ae4a-5494100fa903 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.133491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddde39-48ed-447c-ae4a-5494100fa903 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.133815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddde39-48ed-447c-ae4a-5494100fa903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.136636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bd88e-c1a6-4c4f-900e-cad54ab6d300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:16.137649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bd88e-c1a6-4c4f-900e-cad54ab6d300 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.138150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bd88e-c1a6-4c4f-900e-cad54ab6d300 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.138667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bd88e-c1a6-4c4f-900e-cad54ab6d300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.152207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-01ddde39-48ed-447c-ae4a-5494100fa903 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:16.153577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-01ddde39-48ed-447c-ae4a-5494100fa903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:16.174351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:16.200706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:16.228581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 306/628] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:16.229583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddde39-48ed-447c-ae4a-5494100fa903 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:16.238238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bd88e-c1a6-4c4f-900e-cad54ab6d300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:16.241766 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bd88e-c1a6-4c4f-900e-cad54ab6d300 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:16.352080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddde39-48ed-447c-ae4a-5494100fa903 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:16.361866 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 323/629] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:16.362891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bd88e-c1a6-4c4f-900e-cad54ab6d300 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:16.391108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892fff8-b78f-4acb-84f7-e1178f51af30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:16.391777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892fff8-b78f-4acb-84f7-e1178f51af30 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.392052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892fff8-b78f-4acb-84f7-e1178f51af30 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.392336 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892fff8-b78f-4acb-84f7-e1178f51af30 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.393132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892fff8-b78f-4acb-84f7-e1178f51af30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:16.393476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892fff8-b78f-4acb-84f7-e1178f51af30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:16.394924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892fff8-b78f-4acb-84f7-e1178f51af30 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:16.407732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 307/630] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:16.410567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce1143-968c-4918-a873-06d789fbbb2b None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.411597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce1143-968c-4918-a873-06d789fbbb2b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.412744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce1143-968c-4918-a873-06d789fbbb2b None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.415093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 308/631] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:16.416504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce1143-968c-4918-a873-06d789fbbb2b None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:16.435281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:16.450038 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:16.474815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bd88e-c1a6-4c4f-900e-cad54ab6d300 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:16.511405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48fd20-db46-4726-b351-8571cb1e40fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:16.512198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48fd20-db46-4726-b351-8571cb1e40fc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.512486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48fd20-db46-4726-b351-8571cb1e40fc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.512750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48fd20-db46-4726-b351-8571cb1e40fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.523371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48fd20-db46-4726-b351-8571cb1e40fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:16.526819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce1143-968c-4918-a873-06d789fbbb2b None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-6b48fd20-db46-4726-b351-8571cb1e40fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:16.531147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:16.532087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:16.532946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:16.537436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 324/632] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:16.538613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48fd20-db46-4726-b351-8571cb1e40fc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:16.563197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d853-3854-43fa-bf98-c3dd6a29833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:16.563990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d853-3854-43fa-bf98-c3dd6a29833b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.564276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d853-3854-43fa-bf98-c3dd6a29833b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.564542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d853-3854-43fa-bf98-c3dd6a29833b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.575408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9125d853-3854-43fa-bf98-c3dd6a29833b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:16.577122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9125d853-3854-43fa-bf98-c3dd6a29833b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-9125d853-3854-43fa-bf98-c3dd6a29833b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:16.595499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:16.596632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:16.597624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:16.597624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:16.597624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:16.597624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:16.597624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 309/633] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:16.597904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546ebd6-9b9f-48c3-9370-470f9226f7af None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.598158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546ebd6-9b9f-48c3-9370-470f9226f7af None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.598450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546ebd6-9b9f-48c3-9370-470f9226f7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.614660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-a546ebd6-9b9f-48c3-9370-470f9226f7af None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:16.615770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a546ebd6-9b9f-48c3-9370-470f9226f7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:16.642885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48fd20-db46-4726-b351-8571cb1e40fc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:16.674427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:16.675483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829450b5-5393-4d99-b13e-691c40f74238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:16.676870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829450b5-5393-4d99-b13e-691c40f74238 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.677173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829450b5-5393-4d99-b13e-691c40f74238 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.677449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829450b5-5393-4d99-b13e-691c40f74238 None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/42a5ec793f014cefb9d0a0d7258bbb59/roles/60130d2ba383400091dc5f253d40a43a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.679896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:16.697269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 310/634] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:16.697269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546ebd6-9b9f-48c3-9370-470f9226f7af None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:16.697490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829450b5-5393-4d99-b13e-691c40f74238 None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=42a5ec793f014cefb9d0a0d7258bbb59, role_id=60130d2ba383400091dc5f253d40a43a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:16.697490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829450b5-5393-4d99-b13e-691c40f74238 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:16.780211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546ebd6-9b9f-48c3-9370-470f9226f7af None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:16.810306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 325/635] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:24:16 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/42a5ec793f014cefb9d0a0d7258bbb59/roles/60130d2ba383400091dc5f253d40a43a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:16.811640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74de5e-a4f6-4c82-8b27-314590333af2 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.811924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74de5e-a4f6-4c82-8b27-314590333af2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.812197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74de5e-a4f6-4c82-8b27-314590333af2 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.818098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 326/636] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:16.818098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74de5e-a4f6-4c82-8b27-314590333af2 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:16.825522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3ae10-dc94-4b3f-b4be-30a98eef0586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:16.826514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3ae10-dc94-4b3f-b4be-30a98eef0586 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.826801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3ae10-dc94-4b3f-b4be-30a98eef0586 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.827101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3ae10-dc94-4b3f-b4be-30a98eef0586 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.827893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3ae10-dc94-4b3f-b4be-30a98eef0586 None admin] RBAC: Adding query filter params (name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:16.828205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3ae10-dc94-4b3f-b4be-30a98eef0586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:16.829770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3ae10-dc94-4b3f-b4be-30a98eef0586 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:16.840060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 311/637] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:16.840941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3ae10-dc94-4b3f-b4be-30a98eef0586 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:16.913839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:16.940123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74de5e-a4f6-4c82-8b27-314590333af2 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:16.960787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3ae10-dc94-4b3f-b4be-30a98eef0586 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:16.984008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895e85a-208f-4475-8eb6-b32251561ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:16.984678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895e85a-208f-4475-8eb6-b32251561ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.984957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895e85a-208f-4475-8eb6-b32251561ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.985258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895e85a-208f-4475-8eb6-b32251561ff2 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.985992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895e85a-208f-4475-8eb6-b32251561ff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:16.986292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895e85a-208f-4475-8eb6-b32251561ff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:16.987667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895e85a-208f-4475-8eb6-b32251561ff2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:17.002792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee28209-0685-426a-8142-2fcb0739676c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.003016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 327/638] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:17.003108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28209-0685-426a-8142-2fcb0739676c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.003108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28209-0685-426a-8142-2fcb0739676c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.003108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28209-0685-426a-8142-2fcb0739676c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.003319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbc1f5-bb7c-4c16-8aff-5f4ea70aa506 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.003319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbc1f5-bb7c-4c16-8aff-5f4ea70aa506 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.003319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbc1f5-bb7c-4c16-8aff-5f4ea70aa506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.014806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee28209-0685-426a-8142-2fcb0739676c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:17.020899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-32dbc1f5-bb7c-4c16-8aff-5f4ea70aa506 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:17.022123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-32dbc1f5-bb7c-4c16-8aff-5f4ea70aa506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-bee28209-0685-426a-8142-2fcb0739676c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:17.022787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:17.023722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:17.024553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 312/639] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:17.025376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee28209-0685-426a-8142-2fcb0739676c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.073894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:17.097794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 328/640] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:17.099296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbc1f5-bb7c-4c16-8aff-5f4ea70aa506 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.147860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee28209-0685-426a-8142-2fcb0739676c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:17.169477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:17.187019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d839ca-950c-478f-9ea5-ed1111de03a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.187809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d839ca-950c-478f-9ea5-ed1111de03a4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.188083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d839ca-950c-478f-9ea5-ed1111de03a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.188354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d839ca-950c-478f-9ea5-ed1111de03a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.202166 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d839ca-950c-478f-9ea5-ed1111de03a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:17.204086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d839ca-950c-478f-9ea5-ed1111de03a4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:17.215926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbc1f5-bb7c-4c16-8aff-5f4ea70aa506 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-a7d839ca-950c-478f-9ea5-ed1111de03a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:17.222158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:17.223715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:17.229311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:17.229311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:17.229311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:17.229311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:17.229311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 313/641] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:17.229311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d839ca-950c-478f-9ea5-ed1111de03a4 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.260013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca08cd2-a759-439e-9614-53f9ccdfc3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.260808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca08cd2-a759-439e-9614-53f9ccdfc3ff None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.261089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca08cd2-a759-439e-9614-53f9ccdfc3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.261393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca08cd2-a759-439e-9614-53f9ccdfc3ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.268784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca08cd2-a759-439e-9614-53f9ccdfc3ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:17.270423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca08cd2-a759-439e-9614-53f9ccdfc3ff None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-dca08cd2-a759-439e-9614-53f9ccdfc3ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:17.284408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:17.286173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:17.287085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:17.287085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:17.287085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:17.287085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:17.287085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:17.287085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:17.287085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:17.287085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:17.287085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 329/642] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:17.287085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca08cd2-a759-439e-9614-53f9ccdfc3ff None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.325649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:17.342549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d839ca-950c-478f-9ea5-ed1111de03a4 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:17.379448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27a5e8-1aa4-4afa-b05f-99c5069dc392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.380113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27a5e8-1aa4-4afa-b05f-99c5069dc392 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.380391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27a5e8-1aa4-4afa-b05f-99c5069dc392 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.381096 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27a5e8-1aa4-4afa-b05f-99c5069dc392 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.381884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27a5e8-1aa4-4afa-b05f-99c5069dc392 None admin] RBAC: Adding query filter params (name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:17.382190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27a5e8-1aa4-4afa-b05f-99c5069dc392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:17.383571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27a5e8-1aa4-4afa-b05f-99c5069dc392 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:17.397100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 314/643] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:17.397100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27a5e8-1aa4-4afa-b05f-99c5069dc392 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.397395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca08cd2-a759-439e-9614-53f9ccdfc3ff None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:17.429967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:17.434273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f65e48-545e-4084-8126-8c64a6152536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.435080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f65e48-545e-4084-8126-8c64a6152536 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.435363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f65e48-545e-4084-8126-8c64a6152536 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.435633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f65e48-545e-4084-8126-8c64a6152536 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.446255 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f65e48-545e-4084-8126-8c64a6152536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f2f65e48-545e-4084-8126-8c64a6152536 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:17.454282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:17.455252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:17.456166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 330/644] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:17.456964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f65e48-545e-4084-8126-8c64a6152536 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.514301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27a5e8-1aa4-4afa-b05f-99c5069dc392 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:17.556514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f03689-cade-4f9f-88a3-42986a010876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.557197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f03689-cade-4f9f-88a3-42986a010876 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.557490 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f03689-cade-4f9f-88a3-42986a010876 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.557767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f03689-cade-4f9f-88a3-42986a010876 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.558503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f03689-cade-4f9f-88a3-42986a010876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:17.558813 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f03689-cade-4f9f-88a3-42986a010876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:17.560181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f03689-cade-4f9f-88a3-42986a010876 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:17.561477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:17.568368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f65e48-545e-4084-8126-8c64a6152536 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:17.573382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 315/645] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:17.574125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f03689-cade-4f9f-88a3-42986a010876 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.616418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862eef5-3795-455d-84ac-5a5ace95cf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.617091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862eef5-3795-455d-84ac-5a5ace95cf50 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.617399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862eef5-3795-455d-84ac-5a5ace95cf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.617674 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862eef5-3795-455d-84ac-5a5ace95cf50 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.618461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862eef5-3795-455d-84ac-5a5ace95cf50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:17.618769 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862eef5-3795-455d-84ac-5a5ace95cf50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:17.620199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862eef5-3795-455d-84ac-5a5ace95cf50 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:17.632342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 331/646] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:17.633294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862eef5-3795-455d-84ac-5a5ace95cf50 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.687214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:17.698370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f03689-cade-4f9f-88a3-42986a010876 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:17.739558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862eef5-3795-455d-84ac-5a5ace95cf50 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:17.740539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc28114-398c-44e7-9f82-7cb31bfb98bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.741611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc28114-398c-44e7-9f82-7cb31bfb98bc None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.742752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc28114-398c-44e7-9f82-7cb31bfb98bc None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.743035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc28114-398c-44e7-9f82-7cb31bfb98bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.756137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc28114-398c-44e7-9f82-7cb31bfb98bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:17.757682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc28114-398c-44e7-9f82-7cb31bfb98bc None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:17.775316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37d20-4832-4e3f-b8e5-fa7d80dc2287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.775986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37d20-4832-4e3f-b8e5-fa7d80dc2287 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.776269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37d20-4832-4e3f-b8e5-fa7d80dc2287 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.776544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37d20-4832-4e3f-b8e5-fa7d80dc2287 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-edc28114-398c-44e7-9f82-7cb31bfb98bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:17.776778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:17.777860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:17.778770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:17.778770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:17.778770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:17.778770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:17.779004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37d20-4832-4e3f-b8e5-fa7d80dc2287 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:17.779004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37d20-4832-4e3f-b8e5-fa7d80dc2287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:17.779004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37d20-4832-4e3f-b8e5-fa7d80dc2287 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:17.782556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 316/647] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:17.783421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc28114-398c-44e7-9f82-7cb31bfb98bc None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.792825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 332/648] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:17.792825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37d20-4832-4e3f-b8e5-fa7d80dc2287 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.811333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:17.897661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc28114-398c-44e7-9f82-7cb31bfb98bc None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:17.904441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37d20-4832-4e3f-b8e5-fa7d80dc2287 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:17.938801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46597947-6145-4376-a412-2003310f079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.939505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46597947-6145-4376-a412-2003310f079d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.939784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46597947-6145-4376-a412-2003310f079d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.939963 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503c3b6-115a-4d24-8ae3-fbc4aabd4198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:17.940075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46597947-6145-4376-a412-2003310f079d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.940816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503c3b6-115a-4d24-8ae3-fbc4aabd4198 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:17.941242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503c3b6-115a-4d24-8ae3-fbc4aabd4198 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:17.941524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503c3b6-115a-4d24-8ae3-fbc4aabd4198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:17.943058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503c3b6-115a-4d24-8ae3-fbc4aabd4198 None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=a1a7213cbf6448b7ba6570a52a4c7c0e) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:17.943327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503c3b6-115a-4d24-8ae3-fbc4aabd4198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:17.944818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503c3b6-115a-4d24-8ae3-fbc4aabd4198 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:17.950236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46597947-6145-4376-a412-2003310f079d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:17.961397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 333/649] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/role_assignments?user.id=a1a7213cbf6448b7ba6570a52a4c7c0e&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:17.961397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503c3b6-115a-4d24-8ae3-fbc4aabd4198 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:17.961717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46597947-6145-4376-a412-2003310f079d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-46597947-6145-4376-a412-2003310f079d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:17.991493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:17.992527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:17.993516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:17.993516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:17.993516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:17.993516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:17.993516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:17.993516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:17.993516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:17.993516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:17.993516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:17.993516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:17.997137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 317/650] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:17.997862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46597947-6145-4376-a412-2003310f079d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:18.017118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:18.082241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:18.104866 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503c3b6-115a-4d24-8ae3-fbc4aabd4198 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:18.136412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46597947-6145-4376-a412-2003310f079d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:18.145044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627079-2b8b-4e78-b27d-ff287e125b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:18.145756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627079-2b8b-4e78-b27d-ff287e125b62 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:18.146038 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627079-2b8b-4e78-b27d-ff287e125b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:18.146311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627079-2b8b-4e78-b27d-ff287e125b62 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:18.147045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5627079-2b8b-4e78-b27d-ff287e125b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:18.147351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5627079-2b8b-4e78-b27d-ff287e125b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:18.148834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5627079-2b8b-4e78-b27d-ff287e125b62 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:18.165952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 334/651] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:18.166967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627079-2b8b-4e78-b27d-ff287e125b62 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:18.177587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10d428-87e3-431a-8c08-9259f5e20237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:18.178421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10d428-87e3-431a-8c08-9259f5e20237 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:18.178709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10d428-87e3-431a-8c08-9259f5e20237 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:18.178980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10d428-87e3-431a-8c08-9259f5e20237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:18.192286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10d428-87e3-431a-8c08-9259f5e20237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:18.194042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10d428-87e3-431a-8c08-9259f5e20237 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-8b10d428-87e3-431a-8c08-9259f5e20237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:18.216063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:18.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:18.217957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:18.217957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:18.217957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:18.217957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:18.218810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 318/652] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:18.219655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10d428-87e3-431a-8c08-9259f5e20237 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:18.292485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627079-2b8b-4e78-b27d-ff287e125b62 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:18.300360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:18.333132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003ac1c-76ca-43e9-befc-38bdf14012bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:18.333990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003ac1c-76ca-43e9-befc-38bdf14012bb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:18.334370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003ac1c-76ca-43e9-befc-38bdf14012bb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:18.334812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003ac1c-76ca-43e9-befc-38bdf14012bb None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:18.335721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003ac1c-76ca-43e9-befc-38bdf14012bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:18.336819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003ac1c-76ca-43e9-befc-38bdf14012bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:18.338576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003ac1c-76ca-43e9-befc-38bdf14012bb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:18.348923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:18.364923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 335/653] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jul 14 11:24:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:18.364923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003ac1c-76ca-43e9-befc-38bdf14012bb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:18.375785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10d428-87e3-431a-8c08-9259f5e20237 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:18.423280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79321d9e-42b8-445d-8552-814307b17e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:18.424736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79321d9e-42b8-445d-8552-814307b17e08 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:18.425203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79321d9e-42b8-445d-8552-814307b17e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:18.425650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79321d9e-42b8-445d-8552-814307b17e08 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:18.436805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79321d9e-42b8-445d-8552-814307b17e08 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:18.438533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79321d9e-42b8-445d-8552-814307b17e08 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-79321d9e-42b8-445d-8552-814307b17e08 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:18.460266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:18.461407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:18.462591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:18.462591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:18.462591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:18.462591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:18.462591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:18.462591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:18.462591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:18.462591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:18.462591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 11:24:18.462591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:18.464744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 319/654] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:18.465928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79321d9e-42b8-445d-8552-814307b17e08 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:18.503694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003ac1c-76ca-43e9-befc-38bdf14012bb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:18.540736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60255e-09b9-42d0-8b4f-a7452bffb545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:18.541844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60255e-09b9-42d0-8b4f-a7452bffb545 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:18.542356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60255e-09b9-42d0-8b4f-a7452bffb545 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:18.542862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60255e-09b9-42d0-8b4f-a7452bffb545 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:18.543800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60255e-09b9-42d0-8b4f-a7452bffb545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:18.544312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60255e-09b9-42d0-8b4f-a7452bffb545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:18.545913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60255e-09b9-42d0-8b4f-a7452bffb545 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:18.558866 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 336/655] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:18.560285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60255e-09b9-42d0-8b4f-a7452bffb545 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:18.595898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79321d9e-42b8-445d-8552-814307b17e08 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:18.616476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:18.625316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:18.636293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a92e5e-e763-4e2e-ad5c-99eb1bcc06c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:18.636996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a92e5e-e763-4e2e-ad5c-99eb1bcc06c8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:18.637317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a92e5e-e763-4e2e-ad5c-99eb1bcc06c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:18.637602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a92e5e-e763-4e2e-ad5c-99eb1bcc06c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:18.672323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60255e-09b9-42d0-8b4f-a7452bffb545 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:18.711417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f8ce8-ed57-484f-9343-edb2f4b40794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:18.712498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f8ce8-ed57-484f-9343-edb2f4b40794 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:18.713010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f8ce8-ed57-484f-9343-edb2f4b40794 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:18.713549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f8ce8-ed57-484f-9343-edb2f4b40794 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:18.714789 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f8ce8-ed57-484f-9343-edb2f4b40794 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:18.715344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f8ce8-ed57-484f-9343-edb2f4b40794 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:18.716157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a92e5e-e763-4e2e-ad5c-99eb1bcc06c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:18.717361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f8ce8-ed57-484f-9343-edb2f4b40794 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:18.717716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a92e5e-e763-4e2e-ad5c-99eb1bcc06c8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:18.742575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 337/656] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jul 14 11:24:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:18.743968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f8ce8-ed57-484f-9343-edb2f4b40794 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:18.847482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 320/657] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:18.847482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a92e5e-e763-4e2e-ad5c-99eb1bcc06c8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:18.862221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f8ce8-ed57-484f-9343-edb2f4b40794 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:18.869547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:18.881893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:18.901450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f94380-d630-4f78-86a5-91cffe2a4e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:18.902632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f94380-d630-4f78-86a5-91cffe2a4e74 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:18.903134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f94380-d630-4f78-86a5-91cffe2a4e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:18.903607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f94380-d630-4f78-86a5-91cffe2a4e74 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:18.913624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f94380-d630-4f78-86a5-91cffe2a4e74 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:18.915489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f94380-d630-4f78-86a5-91cffe2a4e74 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f6f94380-d630-4f78-86a5-91cffe2a4e74 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:18.931515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:18.932534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:18.933518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:18.933518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:18.933518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:18.933518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:18.933518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:18.933518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:18.933518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:18.933518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:18.933518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 11:24:18.933518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:18.936507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 338/658] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:18.937690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f94380-d630-4f78-86a5-91cffe2a4e74 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:18.963474 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a92e5e-e763-4e2e-ad5c-99eb1bcc06c8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:18.998816 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b823d0-2b8d-4234-b180-8943ce57f97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:18.999470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b823d0-2b8d-4234-b180-8943ce57f97d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:18.999755 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b823d0-2b8d-4234-b180-8943ce57f97d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.000038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b823d0-2b8d-4234-b180-8943ce57f97d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.048618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f94380-d630-4f78-86a5-91cffe2a4e74 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:19.070154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b823d0-2b8d-4234-b180-8943ce57f97d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:19.071465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b823d0-2b8d-4234-b180-8943ce57f97d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:19.089876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:19.090556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155486d-bedd-4e04-98c2-ce6a86b8f0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:19.091361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155486d-bedd-4e04-98c2-ce6a86b8f0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:19.091639 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155486d-bedd-4e04-98c2-ce6a86b8f0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.091914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155486d-bedd-4e04-98c2-ce6a86b8f0a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.102948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155486d-bedd-4e04-98c2-ce6a86b8f0a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:19.104392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155486d-bedd-4e04-98c2-ce6a86b8f0a2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-a155486d-bedd-4e04-98c2-ce6a86b8f0a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:19.125524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:19.126659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:19.127539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:19.127539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:19.127539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:19.127539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:19.127539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 339/659] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:19.128156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdd0f4-65d1-4ac6-a596-5f77998498b9 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:19.128328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdd0f4-65d1-4ac6-a596-5f77998498b9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.128892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdd0f4-65d1-4ac6-a596-5f77998498b9 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.134437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 340/660] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:19.134437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdd0f4-65d1-4ac6-a596-5f77998498b9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:19.166751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:19.194583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 321/661] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:19.196114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b823d0-2b8d-4234-b180-8943ce57f97d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:19.252289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdd0f4-65d1-4ac6-a596-5f77998498b9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:19.291759 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a5bc3-dc90-4b2b-a56d-51f5162593fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:19.292436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a5bc3-dc90-4b2b-a56d-51f5162593fb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:19.292717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a5bc3-dc90-4b2b-a56d-51f5162593fb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.293287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a5bc3-dc90-4b2b-a56d-51f5162593fb None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.293861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a5bc3-dc90-4b2b-a56d-51f5162593fb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:19.294165 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a5bc3-dc90-4b2b-a56d-51f5162593fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:19.295591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a5bc3-dc90-4b2b-a56d-51f5162593fb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:19.306657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b823d0-2b8d-4234-b180-8943ce57f97d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:19.314283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 341/662] 10.222.0.40 () {56 vars in 1139 bytes} [Thu Jul 14 11:24:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:19.314283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d85d9e-785f-41f7-be31-1c23dd8c3289 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:19.314283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d85d9e-785f-41f7-be31-1c23dd8c3289 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.314283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d85d9e-785f-41f7-be31-1c23dd8c3289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.329569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:19.333864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-46d85d9e-785f-41f7-be31-1c23dd8c3289 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:19.335069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-46d85d9e-785f-41f7-be31-1c23dd8c3289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:19.349454 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a24eab-f97f-4143-9dbc-ff6ead5a3819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:19.351851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a24eab-f97f-4143-9dbc-ff6ead5a3819 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:19.351851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a24eab-f97f-4143-9dbc-ff6ead5a3819 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.351851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a24eab-f97f-4143-9dbc-ff6ead5a3819 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.351851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a24eab-f97f-4143-9dbc-ff6ead5a3819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:19.351851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a24eab-f97f-4143-9dbc-ff6ead5a3819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:19.353108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a24eab-f97f-4143-9dbc-ff6ead5a3819 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:19.370245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 322/663] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:19.370245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a24eab-f97f-4143-9dbc-ff6ead5a3819 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:19.417091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 342/664] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:19.418316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d85d9e-785f-41f7-be31-1c23dd8c3289 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:19.440277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:19.480914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a24eab-f97f-4143-9dbc-ff6ead5a3819 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:19.516139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f2b22-8847-4402-add7-d9f2d6da55e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:19.516924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f2b22-8847-4402-add7-d9f2d6da55e7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:19.517233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f2b22-8847-4402-add7-d9f2d6da55e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.517510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f2b22-8847-4402-add7-d9f2d6da55e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.520278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d85d9e-785f-41f7-be31-1c23dd8c3289 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:19.527924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f2b22-8847-4402-add7-d9f2d6da55e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:19.529409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f2b22-8847-4402-add7-d9f2d6da55e7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-014f2b22-8847-4402-add7-d9f2d6da55e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:19.548008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:19.549130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:19.550058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:19.550058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:19.550058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:19.550058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:19.550058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 323/665] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:19.550058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f2b22-8847-4402-add7-d9f2d6da55e7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:19.564570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c4073-9a65-4707-ab37-6e21c257b4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:19.565376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c4073-9a65-4707-ab37-6e21c257b4bb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:19.565651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c4073-9a65-4707-ab37-6e21c257b4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.565919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c4073-9a65-4707-ab37-6e21c257b4bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.589328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:19.638530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c4073-9a65-4707-ab37-6e21c257b4bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:19.640133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c4073-9a65-4707-ab37-6e21c257b4bb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:19.659325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:19.667259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f2b22-8847-4402-add7-d9f2d6da55e7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:19.702875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed23dae-bcc1-43aa-ac4a-f038eb6be803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:19.703669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed23dae-bcc1-43aa-ac4a-f038eb6be803 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:19.703952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed23dae-bcc1-43aa-ac4a-f038eb6be803 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.704232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed23dae-bcc1-43aa-ac4a-f038eb6be803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.712210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed23dae-bcc1-43aa-ac4a-f038eb6be803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:19.713716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed23dae-bcc1-43aa-ac4a-f038eb6be803 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-fed23dae-bcc1-43aa-ac4a-f038eb6be803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:19.728287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:19.729396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:19.730464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:19.730464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:19.730464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:19.730464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:19.730464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:19.730464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:19.730464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:19.730464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:19.734161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 324/666] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:19.735002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed23dae-bcc1-43aa-ac4a-f038eb6be803 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:19.753292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 343/667] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:19.753292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c4073-9a65-4707-ab37-6e21c257b4bb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:19.838302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:19.850777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed23dae-bcc1-43aa-ac4a-f038eb6be803 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:19.867440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c4073-9a65-4707-ab37-6e21c257b4bb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:19.883782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:19.886167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d43012-ea33-411b-8372-7361d2ab1d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:19.887142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d43012-ea33-411b-8372-7361d2ab1d24 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:19.887616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d43012-ea33-411b-8372-7361d2ab1d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.887892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d43012-ea33-411b-8372-7361d2ab1d24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.898915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d43012-ea33-411b-8372-7361d2ab1d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:19.901069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b277a-1a01-4d29-9138-9cede8a01f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:19.901753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b277a-1a01-4d29-9138-9cede8a01f9a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:19.902032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b277a-1a01-4d29-9138-9cede8a01f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:19.902301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b277a-1a01-4d29-9138-9cede8a01f9a None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:19.903025 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b277a-1a01-4d29-9138-9cede8a01f9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:19.903326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b277a-1a01-4d29-9138-9cede8a01f9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:19.904675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b277a-1a01-4d29-9138-9cede8a01f9a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-76d43012-ea33-411b-8372-7361d2ab1d24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:19.906650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:19.907650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:19.908491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 325/668] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:19.909230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d43012-ea33-411b-8372-7361d2ab1d24 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:19.927205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 344/669] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:19.927205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b277a-1a01-4d29-9138-9cede8a01f9a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.028804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d43012-ea33-411b-8372-7361d2ab1d24 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.029970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b277a-1a01-4d29-9138-9cede8a01f9a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.063254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1b11a-f957-4c3d-a9a6-5fdfddd1e528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.063832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b11a-f957-4c3d-a9a6-5fdfddd1e528 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.064114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b11a-f957-4c3d-a9a6-5fdfddd1e528 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.064392 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b11a-f957-4c3d-a9a6-5fdfddd1e528 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.065204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1b11a-f957-4c3d-a9a6-5fdfddd1e528 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:20.065554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1b11a-f957-4c3d-a9a6-5fdfddd1e528 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.066307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00897fa-3c30-4963-8e70-5deb4ed7d714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.067476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1b11a-f957-4c3d-a9a6-5fdfddd1e528 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.067946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00897fa-3c30-4963-8e70-5deb4ed7d714 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.068383 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00897fa-3c30-4963-8e70-5deb4ed7d714 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.068793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00897fa-3c30-4963-8e70-5deb4ed7d714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.080059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00897fa-3c30-4963-8e70-5deb4ed7d714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.080880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 326/670] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:20.081742 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1b11a-f957-4c3d-a9a6-5fdfddd1e528 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.082166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00897fa-3c30-4963-8e70-5deb4ed7d714 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.094399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c00897fa-3c30-4963-8e70-5deb4ed7d714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:20.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:20.107795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:20.108724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:20.108724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:20.108724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:20.108724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:20.108724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 345/671] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:20.108724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00897fa-3c30-4963-8e70-5deb4ed7d714 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.140619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:20.188871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1b11a-f957-4c3d-a9a6-5fdfddd1e528 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.221663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00897fa-3c30-4963-8e70-5deb4ed7d714 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.225797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721faf1-f8ab-44ac-9a8c-5437b0334326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.226472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721faf1-f8ab-44ac-9a8c-5437b0334326 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.226755 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721faf1-f8ab-44ac-9a8c-5437b0334326 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.227029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721faf1-f8ab-44ac-9a8c-5437b0334326 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.227782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721faf1-f8ab-44ac-9a8c-5437b0334326 None admin] RBAC: Adding query filter params (name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:20.228088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721faf1-f8ab-44ac-9a8c-5437b0334326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.229490 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721faf1-f8ab-44ac-9a8c-5437b0334326 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.237604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 327/672] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:20.239051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721faf1-f8ab-44ac-9a8c-5437b0334326 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.259003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13152c48-0b51-4179-9e1e-664e3a91384b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.259767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13152c48-0b51-4179-9e1e-664e3a91384b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.260105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13152c48-0b51-4179-9e1e-664e3a91384b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.260296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13152c48-0b51-4179-9e1e-664e3a91384b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.267032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13152c48-0b51-4179-9e1e-664e3a91384b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.268469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13152c48-0b51-4179-9e1e-664e3a91384b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-13152c48-0b51-4179-9e1e-664e3a91384b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:20.282391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:20.283391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:20.284299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:20.284299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:20.284299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:20.284299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:20.284299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:20.284299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:20.284299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:20.284299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:20.284299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 346/673] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:20.284299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13152c48-0b51-4179-9e1e-664e3a91384b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.309257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:20.340292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721faf1-f8ab-44ac-9a8c-5437b0334326 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.372995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:20.374144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8496f2-546c-4c0e-8c23-5f5fb1bea798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.374802 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8496f2-546c-4c0e-8c23-5f5fb1bea798 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.375116 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8496f2-546c-4c0e-8c23-5f5fb1bea798 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.375368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8496f2-546c-4c0e-8c23-5f5fb1bea798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.376202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8496f2-546c-4c0e-8c23-5f5fb1bea798 None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=147d764ef17042dbb7ea179c29711588) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:20.376515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8496f2-546c-4c0e-8c23-5f5fb1bea798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.377989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8496f2-546c-4c0e-8c23-5f5fb1bea798 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.384514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 328/674] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/role_assignments?user.id=147d764ef17042dbb7ea179c29711588&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:20.385282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13152c48-0b51-4179-9e1e-664e3a91384b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.390194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866878d-d684-4817-8e86-996bca0575c3 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.390194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866878d-d684-4817-8e86-996bca0575c3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.390194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866878d-d684-4817-8e86-996bca0575c3 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.390194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 329/675] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:20.390194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866878d-d684-4817-8e86-996bca0575c3 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.424593 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be52ea-aa47-4e42-b4a6-aa30062e34e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.425358 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be52ea-aa47-4e42-b4a6-aa30062e34e3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.425712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be52ea-aa47-4e42-b4a6-aa30062e34e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.425974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be52ea-aa47-4e42-b4a6-aa30062e34e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.427333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be52ea-aa47-4e42-b4a6-aa30062e34e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:20.427692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be52ea-aa47-4e42-b4a6-aa30062e34e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.429109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be52ea-aa47-4e42-b4a6-aa30062e34e3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.444885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 347/676] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:20.445968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be52ea-aa47-4e42-b4a6-aa30062e34e3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.523814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866878d-d684-4817-8e86-996bca0575c3 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.558505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be52ea-aa47-4e42-b4a6-aa30062e34e3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.564447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e76c1-2a7f-4ea3-b7bd-ae7d70d48896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.565657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e76c1-2a7f-4ea3-b7bd-ae7d70d48896 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.565657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e76c1-2a7f-4ea3-b7bd-ae7d70d48896 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.565865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e76c1-2a7f-4ea3-b7bd-ae7d70d48896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.568407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:20.577740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e76c1-2a7f-4ea3-b7bd-ae7d70d48896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.580604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e76c1-2a7f-4ea3-b7bd-ae7d70d48896 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.600177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b1d21-789c-4d2a-8870-9bfbaf57592c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.601131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b1d21-789c-4d2a-8870-9bfbaf57592c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.601493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b1d21-789c-4d2a-8870-9bfbaf57592c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.601775 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b1d21-789c-4d2a-8870-9bfbaf57592c None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.602582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b1d21-789c-4d2a-8870-9bfbaf57592c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-cc9e76c1-2a7f-4ea3-b7bd-ae7d70d48896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:20.602723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:20.603630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:20.604612 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:20.604612 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:20.604612 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:20.604612 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:20.604868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b1d21-789c-4d2a-8870-9bfbaf57592c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.604947 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 330/677] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:20.605512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b812d-f8cd-42a0-8565-27a3e9de7f12 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.607907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b812d-f8cd-42a0-8565-27a3e9de7f12 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.607907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b812d-f8cd-42a0-8565-27a3e9de7f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.608810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b1d21-789c-4d2a-8870-9bfbaf57592c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.622524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 348/678] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:20.623301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b1d21-789c-4d2a-8870-9bfbaf57592c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.624838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-2f0b812d-f8cd-42a0-8565-27a3e9de7f12 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:20.626088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2f0b812d-f8cd-42a0-8565-27a3e9de7f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:20.635382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:20.699108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 331/679] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:20.700081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b812d-f8cd-42a0-8565-27a3e9de7f12 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.736067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b1d21-789c-4d2a-8870-9bfbaf57592c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.770857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd850e-1791-49d8-ac07-2b9f2d6e9685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.771658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd850e-1791-49d8-ac07-2b9f2d6e9685 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.771937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd850e-1791-49d8-ac07-2b9f2d6e9685 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.772209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd850e-1791-49d8-ac07-2b9f2d6e9685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.780023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd850e-1791-49d8-ac07-2b9f2d6e9685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.781498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd850e-1791-49d8-ac07-2b9f2d6e9685 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b9fd850e-1791-49d8-ac07-2b9f2d6e9685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:20.796026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:20.797036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:20.798060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:20.798060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:20.798060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:20.798060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:20.798060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:20.798060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:20.798060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:20.798060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:20.798060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 349/680] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:20.798611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd850e-1791-49d8-ac07-2b9f2d6e9685 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.810759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b812d-f8cd-42a0-8565-27a3e9de7f12 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.850108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69946a45-1b2e-4417-bc86-65e226a71020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.850867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69946a45-1b2e-4417-bc86-65e226a71020 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.851120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69946a45-1b2e-4417-bc86-65e226a71020 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.851615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69946a45-1b2e-4417-bc86-65e226a71020 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.854284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:20.855108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69946a45-1b2e-4417-bc86-65e226a71020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:20.856021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69946a45-1b2e-4417-bc86-65e226a71020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.857466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69946a45-1b2e-4417-bc86-65e226a71020 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.869114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 332/681] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:20.869987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69946a45-1b2e-4417-bc86-65e226a71020 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:20.885426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:20.917627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd850e-1791-49d8-ac07-2b9f2d6e9685 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.956807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dee309-a1f5-4ec5-8dc4-936fcf9007ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:20.958340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dee309-a1f5-4ec5-8dc4-936fcf9007ec None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:20.958624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dee309-a1f5-4ec5-8dc4-936fcf9007ec None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:20.962371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dee309-a1f5-4ec5-8dc4-936fcf9007ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:20.962371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dee309-a1f5-4ec5-8dc4-936fcf9007ec None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=f12e348368ac42c696623b4a8f781977) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:20.962371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dee309-a1f5-4ec5-8dc4-936fcf9007ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:20.962371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dee309-a1f5-4ec5-8dc4-936fcf9007ec None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:20.978654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69946a45-1b2e-4417-bc86-65e226a71020 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:20.990257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 350/682] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/role_assignments?user.id=f12e348368ac42c696623b4a8f781977&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:20.991115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dee309-a1f5-4ec5-8dc4-936fcf9007ec None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:21.033332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2878d0-750d-416c-b86e-71118ac304de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:21.034506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2878d0-750d-416c-b86e-71118ac304de None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:21.034988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2878d0-750d-416c-b86e-71118ac304de None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:21.035482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2878d0-750d-416c-b86e-71118ac304de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:21.050095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2878d0-750d-416c-b86e-71118ac304de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:21.051827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2878d0-750d-416c-b86e-71118ac304de None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-fe2878d0-750d-416c-b86e-71118ac304de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:21.072141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:21.073451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:21.074588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:21.074588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:21.074588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:21.074588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:21.081091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 333/683] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:21.082937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2878d0-750d-416c-b86e-71118ac304de None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:21.118324 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:21.130370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dee309-a1f5-4ec5-8dc4-936fcf9007ec None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:21.181683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:21.198762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22638f9-84b5-4c92-a7b7-b50572cb5cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:21.200075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22638f9-84b5-4c92-a7b7-b50572cb5cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:21.200761 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22638f9-84b5-4c92-a7b7-b50572cb5cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:21.201184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22638f9-84b5-4c92-a7b7-b50572cb5cd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:21.202118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22638f9-84b5-4c92-a7b7-b50572cb5cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:21.202557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22638f9-84b5-4c92-a7b7-b50572cb5cd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:21.204522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22638f9-84b5-4c92-a7b7-b50572cb5cd8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:21.227102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2878d0-750d-416c-b86e-71118ac304de None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:21.232446 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 351/684] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:21.233607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22638f9-84b5-4c92-a7b7-b50572cb5cd8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:21.285289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60d212-eecd-4da7-b35f-c3ee73654f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:21.286625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60d212-eecd-4da7-b35f-c3ee73654f2e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:21.286906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60d212-eecd-4da7-b35f-c3ee73654f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:21.287175 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60d212-eecd-4da7-b35f-c3ee73654f2e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:21.295530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60d212-eecd-4da7-b35f-c3ee73654f2e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:21.296994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60d212-eecd-4da7-b35f-c3ee73654f2e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-ce60d212-eecd-4da7-b35f-c3ee73654f2e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:21.312512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:21.313567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:21.314531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:21.314531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:21.314531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:21.314531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:21.314531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:21.314531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:21.314531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:21.314531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:21.314531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:21.314531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:21.318672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 334/685] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:21.319598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60d212-eecd-4da7-b35f-c3ee73654f2e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:21.388974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22638f9-84b5-4c92-a7b7-b50572cb5cd8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:21.437605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:21.443662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a45674-2d85-49a1-9123-dbafd799b2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:21.444778 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a45674-2d85-49a1-9123-dbafd799b2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:21.445565 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a45674-2d85-49a1-9123-dbafd799b2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:21.446408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a45674-2d85-49a1-9123-dbafd799b2b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:21.448345 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a45674-2d85-49a1-9123-dbafd799b2b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:21.449360 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a45674-2d85-49a1-9123-dbafd799b2b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:21.459983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60d212-eecd-4da7-b35f-c3ee73654f2e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:21.460125 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a45674-2d85-49a1-9123-dbafd799b2b0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:21.476931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 352/686] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:21.479151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a45674-2d85-49a1-9123-dbafd799b2b0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:21.497506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbece7-0083-4934-8ea8-cc32e0de0123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:21.498267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbece7-0083-4934-8ea8-cc32e0de0123 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:21.498646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbece7-0083-4934-8ea8-cc32e0de0123 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:21.499843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbece7-0083-4934-8ea8-cc32e0de0123 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:21.501081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbece7-0083-4934-8ea8-cc32e0de0123 None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=481ef56b1cfa465a8efc59736a34e412) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:21.502154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbece7-0083-4934-8ea8-cc32e0de0123 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:21.503866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbece7-0083-4934-8ea8-cc32e0de0123 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:21.518937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 335/687] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:21 2022] GET /identity/v3/role_assignments?user.id=481ef56b1cfa465a8efc59736a34e412&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:21.521356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbece7-0083-4934-8ea8-cc32e0de0123 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:21.605842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:21.675455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a45674-2d85-49a1-9123-dbafd799b2b0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:21.715460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb211532-3460-45c3-8851-878ed32f20f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:21.721188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb211532-3460-45c3-8851-878ed32f20f1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:21.721188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb211532-3460-45c3-8851-878ed32f20f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:21.721188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb211532-3460-45c3-8851-878ed32f20f1 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:21.721188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb211532-3460-45c3-8851-878ed32f20f1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:21.721188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb211532-3460-45c3-8851-878ed32f20f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:21.721188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb211532-3460-45c3-8851-878ed32f20f1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:21.741834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 353/688] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jul 14 11:24:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:21.749187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb211532-3460-45c3-8851-878ed32f20f1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:21.812463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbece7-0083-4934-8ea8-cc32e0de0123 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:21.883523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30b6da-64ca-469a-80a4-d188f72b2d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:21.884618 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30b6da-64ca-469a-80a4-d188f72b2d7b None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:21.885191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30b6da-64ca-469a-80a4-d188f72b2d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:21.885726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30b6da-64ca-469a-80a4-d188f72b2d7b None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:21.886653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30b6da-64ca-469a-80a4-d188f72b2d7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:21.891151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30b6da-64ca-469a-80a4-d188f72b2d7b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:21.909590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb211532-3460-45c3-8851-878ed32f20f1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:21.922299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:21.946798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb49832-5cf6-48f1-8d9d-86af51c9f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:21.947769 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb49832-5cf6-48f1-8d9d-86af51c9f35e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:21.948277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb49832-5cf6-48f1-8d9d-86af51c9f35e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:21.948754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb49832-5cf6-48f1-8d9d-86af51c9f35e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:21.991984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 336/689] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:24:21 2022] POST /identity/v3/users => generated 254 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:22.022708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb49832-5cf6-48f1-8d9d-86af51c9f35e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:22.024531 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb49832-5cf6-48f1-8d9d-86af51c9f35e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:22.134652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 354/690] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:22.140539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30b6da-64ca-469a-80a4-d188f72b2d7b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:22.165572 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:22.269260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30b6da-64ca-469a-80a4-d188f72b2d7b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:22.305274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c500e19-88f2-4945-b8f7-588fcf6526bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:22.306464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c500e19-88f2-4945-b8f7-588fcf6526bf None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:22.306941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c500e19-88f2-4945-b8f7-588fcf6526bf None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:22.307413 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c500e19-88f2-4945-b8f7-588fcf6526bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:22.319751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c500e19-88f2-4945-b8f7-588fcf6526bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:22.321531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c500e19-88f2-4945-b8f7-588fcf6526bf None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-2c500e19-88f2-4945-b8f7-588fcf6526bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:22.340249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:22.341196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:22.342107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:22.342107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:22.342107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:22.342107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:22.344355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 337/691] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:22.350145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb49832-5cf6-48f1-8d9d-86af51c9f35e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:22.362412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:22.456052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb49832-5cf6-48f1-8d9d-86af51c9f35e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:22.494940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b097f-6498-4a8e-b420-db7da36247bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:22.496003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b097f-6498-4a8e-b420-db7da36247bc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:22.496511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b097f-6498-4a8e-b420-db7da36247bc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:22.496978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b097f-6498-4a8e-b420-db7da36247bc None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:22.498005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b097f-6498-4a8e-b420-db7da36247bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:22.498508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b097f-6498-4a8e-b420-db7da36247bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:22.500178 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b097f-6498-4a8e-b420-db7da36247bc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:22.513769 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 355/692] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:22.519595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c500e19-88f2-4945-b8f7-588fcf6526bf None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:22.581375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:22.631587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c500e19-88f2-4945-b8f7-588fcf6526bf None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:22.672650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245181ed-bd10-4b44-b025-8921cf7ef8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:22.673480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245181ed-bd10-4b44-b025-8921cf7ef8de None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:22.673765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245181ed-bd10-4b44-b025-8921cf7ef8de None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:22.674039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245181ed-bd10-4b44-b025-8921cf7ef8de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:22.681693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245181ed-bd10-4b44-b025-8921cf7ef8de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:22.683155 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245181ed-bd10-4b44-b025-8921cf7ef8de None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-245181ed-bd10-4b44-b025-8921cf7ef8de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:22.704410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:22.705493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:22.706420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:22.706420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:22.706420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:22.706420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:22.706420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:22.706420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:22.706420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:22.706420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:22.706420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 338/693] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:22.712149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b097f-6498-4a8e-b420-db7da36247bc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:22.803814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99def5-cddd-49af-91db-9b7b01457682 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:22.804125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99def5-cddd-49af-91db-9b7b01457682 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:22.804739 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99def5-cddd-49af-91db-9b7b01457682 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:22.809264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 339/694] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:24:22.811211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f94a7-a08e-46fe-908f-024424b8692f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:22.811503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f94a7-a08e-46fe-908f-024424b8692f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:22.811786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f94a7-a08e-46fe-908f-024424b8692f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:22.835635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c13f94a7-a08e-46fe-908f-024424b8692f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:22.838177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c13f94a7-a08e-46fe-908f-024424b8692f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:22.860864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:22.922714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b097f-6498-4a8e-b420-db7da36247bc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:22.970605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fcc7f-718d-4f90-ac22-8a55ccaf3cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:22.971297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fcc7f-718d-4f90-ac22-8a55ccaf3cab None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:22.971576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fcc7f-718d-4f90-ac22-8a55ccaf3cab None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:22.971848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fcc7f-718d-4f90-ac22-8a55ccaf3cab None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:22.972649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fcc7f-718d-4f90-ac22-8a55ccaf3cab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:22.973530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fcc7f-718d-4f90-ac22-8a55ccaf3cab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:22.974989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fcc7f-718d-4f90-ac22-8a55ccaf3cab None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:22.979770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 340/695] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:24:22.999568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 356/696] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:23.003451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f94a7-a08e-46fe-908f-024424b8692f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:23.005235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fcc7f-718d-4f90-ac22-8a55ccaf3cab None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:23.078733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:23.168224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f94a7-a08e-46fe-908f-024424b8692f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:23.192978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fcc7f-718d-4f90-ac22-8a55ccaf3cab None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:23.243030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6255c94-84a8-4d90-9715-d01d9a02c5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:23.243902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6255c94-84a8-4d90-9715-d01d9a02c5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:23.244242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6255c94-84a8-4d90-9715-d01d9a02c5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:23.244523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6255c94-84a8-4d90-9715-d01d9a02c5a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:23.257400 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:23.261082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6255c94-84a8-4d90-9715-d01d9a02c5a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-a6255c94-84a8-4d90-9715-d01d9a02c5a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:23.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:23.278673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:23.280686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:23.282314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bbeb8-e70e-49a7-9b79-44b18bddbaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:23.282314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bbeb8-e70e-49a7-9b79-44b18bddbaaf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:23.282314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bbeb8-e70e-49a7-9b79-44b18bddbaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:23.282314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bbeb8-e70e-49a7-9b79-44b18bddbaaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:23.282934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bbeb8-e70e-49a7-9b79-44b18bddbaaf None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=42a5ec793f014cefb9d0a0d7258bbb59) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:23.282934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bbeb8-e70e-49a7-9b79-44b18bddbaaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:23.282934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bbeb8-e70e-49a7-9b79-44b18bddbaaf None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:23.286382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 341/697] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:23.288097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6255c94-84a8-4d90-9715-d01d9a02c5a5 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:23.293207 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 357/698] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:23 2022] GET /identity/v3/role_assignments?user.id=42a5ec793f014cefb9d0a0d7258bbb59&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:23.479951 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6255c94-84a8-4d90-9715-d01d9a02c5a5 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:23.491603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:23.516473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e48b2-5e38-4def-8184-6ba3eb0446c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:23.517206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e48b2-5e38-4def-8184-6ba3eb0446c6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:23.517494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e48b2-5e38-4def-8184-6ba3eb0446c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:23.517764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e48b2-5e38-4def-8184-6ba3eb0446c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:23.518515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e48b2-5e38-4def-8184-6ba3eb0446c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:23.518814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e48b2-5e38-4def-8184-6ba3eb0446c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:23.520846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e48b2-5e38-4def-8184-6ba3eb0446c6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:23.530139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 342/699] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:23.536069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bbeb8-e70e-49a7-9b79-44b18bddbaaf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:23.661093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bbeb8-e70e-49a7-9b79-44b18bddbaaf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:23.699972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2656c8-5475-48fb-a8b5-46126f6196bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:23.701045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2656c8-5475-48fb-a8b5-46126f6196bd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:23.702180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2656c8-5475-48fb-a8b5-46126f6196bd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:23.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16577a7e-3cf8-45c0-aa5c-80390dd5b9ba None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:23.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16577a7e-3cf8-45c0-aa5c-80390dd5b9ba None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:23.703659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2656c8-5475-48fb-a8b5-46126f6196bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:23.703820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16577a7e-3cf8-45c0-aa5c-80390dd5b9ba None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:23.709535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 343/700] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:24:23.711858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3cc1a-b75a-4201-9dd4-4bc1e1b00d06 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:23.711858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3cc1a-b75a-4201-9dd4-4bc1e1b00d06 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:23.712241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3cc1a-b75a-4201-9dd4-4bc1e1b00d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:23.715872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2656c8-5475-48fb-a8b5-46126f6196bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:23.718452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2656c8-5475-48fb-a8b5-46126f6196bd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:23.733474 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-2ca3cc1a-b75a-4201-9dd4-4bc1e1b00d06 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:23.735258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2ca3cc1a-b75a-4201-9dd4-4bc1e1b00d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3c2656c8-5475-48fb-a8b5-46126f6196bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:23.750040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:23.752553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:23.755289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:23.755289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:23.755289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:23.755289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:23.755289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 358/701] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:23.758746 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2656c8-5475-48fb-a8b5-46126f6196bd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:23.833401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 344/702] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 11:24:23.837566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:23.871377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3cc1a-b75a-4201-9dd4-4bc1e1b00d06 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:23.927138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2656c8-5475-48fb-a8b5-46126f6196bd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:24.001047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be99d2d-8e49-4a36-a729-76ed2d7b2511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:24.001675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99d2d-8e49-4a36-a729-76ed2d7b2511 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.001953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99d2d-8e49-4a36-a729-76ed2d7b2511 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.002226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99d2d-8e49-4a36-a729-76ed2d7b2511 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.002969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be99d2d-8e49-4a36-a729-76ed2d7b2511 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:24.003274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be99d2d-8e49-4a36-a729-76ed2d7b2511 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:24.009367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be99d2d-8e49-4a36-a729-76ed2d7b2511 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:24.032290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 359/703] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:24.042192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be99d2d-8e49-4a36-a729-76ed2d7b2511 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:24.053848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3cc1a-b75a-4201-9dd4-4bc1e1b00d06 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:24.057493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:24.119259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329dc1f-078e-4115-a149-e719eaa0081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:24.120806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329dc1f-078e-4115-a149-e719eaa0081b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.121366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329dc1f-078e-4115-a149-e719eaa0081b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.121870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329dc1f-078e-4115-a149-e719eaa0081b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.123303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329dc1f-078e-4115-a149-e719eaa0081b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:24.126288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329dc1f-078e-4115-a149-e719eaa0081b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:24.145886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be99d2d-8e49-4a36-a729-76ed2d7b2511 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:24.155287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-9329dc1f-078e-4115-a149-e719eaa0081b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:24.156903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:24.158120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 11:24:24.159261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:24.162196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 345/704] 10.222.0.40 () {56 vars in 1129 bytes} [Thu Jul 14 11:24:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:24.163955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afedce-7714-420c-9999-33ebdb7ceac8 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.164460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afedce-7714-420c-9999-33ebdb7ceac8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.164941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afedce-7714-420c-9999-33ebdb7ceac8 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.166851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 346/705] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:24.167853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afedce-7714-420c-9999-33ebdb7ceac8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:24.183882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46947c-7216-4a46-92f5-a2b34a1a6ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:24.184556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46947c-7216-4a46-92f5-a2b34a1a6ecc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.185067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46947c-7216-4a46-92f5-a2b34a1a6ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.185067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46947c-7216-4a46-92f5-a2b34a1a6ecc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.193671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46947c-7216-4a46-92f5-a2b34a1a6ecc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:24.195144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46947c-7216-4a46-92f5-a2b34a1a6ecc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-dd46947c-7216-4a46-92f5-a2b34a1a6ecc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:24.211334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:24.213973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:24.216014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:24.216014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:24.216014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:24.216014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:24.216014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:24.216014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:24.216014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:24.216014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:24.216014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:24.216014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:24.218166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 360/706] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jul 14 11:24:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:24.218648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6338a-b7c4-485c-ae97-91fce55fe72d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.219282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6338a-b7c4-485c-ae97-91fce55fe72d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.220088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6338a-b7c4-485c-ae97-91fce55fe72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.237803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2df6338a-b7c4-485c-ae97-91fce55fe72d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:24.239775 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2df6338a-b7c4-485c-ae97-91fce55fe72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:24.310758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afedce-7714-420c-9999-33ebdb7ceac8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:24.329924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 361/707] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:24.330994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6338a-b7c4-485c-ae97-91fce55fe72d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:24.465288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de70a1-010c-40fc-baec-1fac90edf446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:24.465964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:24.466649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de70a1-010c-40fc-baec-1fac90edf446 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.466649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de70a1-010c-40fc-baec-1fac90edf446 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.466991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de70a1-010c-40fc-baec-1fac90edf446 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.467839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de70a1-010c-40fc-baec-1fac90edf446 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:24.468244 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de70a1-010c-40fc-baec-1fac90edf446 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:24.469819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de70a1-010c-40fc-baec-1fac90edf446 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:24.474004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6338a-b7c4-485c-ae97-91fce55fe72d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:24.482488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 347/708] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jul 14 11:24:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:24.482488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de70a1-010c-40fc-baec-1fac90edf446 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:24.514354 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7097a1b-f47a-40a6-bf5a-315b67ae9ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:24.517602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7097a1b-f47a-40a6-bf5a-315b67ae9ecc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.517602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7097a1b-f47a-40a6-bf5a-315b67ae9ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.517602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7097a1b-f47a-40a6-bf5a-315b67ae9ecc None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.517602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7097a1b-f47a-40a6-bf5a-315b67ae9ecc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:24.517602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7097a1b-f47a-40a6-bf5a-315b67ae9ecc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:24.518267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:24.523470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7097a1b-f47a-40a6-bf5a-315b67ae9ecc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:24.541913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 362/709] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jul 14 11:24:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:24.543693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7097a1b-f47a-40a6-bf5a-315b67ae9ecc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:24.601327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de70a1-010c-40fc-baec-1fac90edf446 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:24.642537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68715a3-8b5f-4755-af77-64f0cafc39db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:24.643358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68715a3-8b5f-4755-af77-64f0cafc39db None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.643641 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68715a3-8b5f-4755-af77-64f0cafc39db None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.643916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68715a3-8b5f-4755-af77-64f0cafc39db None admin] PATH_INFO: `/v3/services/network` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.644614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68715a3-8b5f-4755-af77-64f0cafc39db None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:24.646037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68715a3-8b5f-4755-af77-64f0cafc39db None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-e68715a3-8b5f-4755-af77-64f0cafc39db None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:24.659015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:24.660812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 11:24:24.661704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:24.665508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 348/710] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:24 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:24.666370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68715a3-8b5f-4755-af77-64f0cafc39db None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:24.681760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7097a1b-f47a-40a6-bf5a-315b67ae9ecc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:24.729700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950e80f-9c13-408f-9331-5d28e97e18e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:24.731285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950e80f-9c13-408f-9331-5d28e97e18e3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.731842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950e80f-9c13-408f-9331-5d28e97e18e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.732278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950e80f-9c13-408f-9331-5d28e97e18e3 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.733452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950e80f-9c13-408f-9331-5d28e97e18e3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:24.734341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950e80f-9c13-408f-9331-5d28e97e18e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:24.736162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950e80f-9c13-408f-9331-5d28e97e18e3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:24.745658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 363/711] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jul 14 11:24:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:24.746784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950e80f-9c13-408f-9331-5d28e97e18e3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:24.801935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:24.802701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:24.811307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68715a3-8b5f-4755-af77-64f0cafc39db None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:24.854881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98814f8-161c-4921-9209-11422013e95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:24.855643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98814f8-161c-4921-9209-11422013e95c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.855911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98814f8-161c-4921-9209-11422013e95c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.856182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98814f8-161c-4921-9209-11422013e95c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.889235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950e80f-9c13-408f-9331-5d28e97e18e3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:24.928422 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc24ab-0c58-4f29-b016-da8f3304bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:24.929083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc24ab-0c58-4f29-b016-da8f3304bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.929389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc24ab-0c58-4f29-b016-da8f3304bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.929659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc24ab-0c58-4f29-b016-da8f3304bfb8 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.930361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc24ab-0c58-4f29-b016-da8f3304bfb8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:24.930660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc24ab-0c58-4f29-b016-da8f3304bfb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:24.932016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc24ab-0c58-4f29-b016-da8f3304bfb8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:24.936938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 364/712] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:24.938977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330ee28-67b0-4497-9462-2b9fa368c060 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:24.939264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330ee28-67b0-4497-9462-2b9fa368c060 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:24.939536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330ee28-67b0-4497-9462-2b9fa368c060 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:24.940901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 365/713] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:24.944700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330ee28-67b0-4497-9462-2b9fa368c060 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:24.953778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98814f8-161c-4921-9209-11422013e95c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:24.955577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98814f8-161c-4921-9209-11422013e95c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:25.056726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:25.066035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:25.074349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 349/714] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:25.076104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017642c7-d922-4618-9aa9-9acd500d456c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:25.076486 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017642c7-d922-4618-9aa9-9acd500d456c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:25.076850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017642c7-d922-4618-9aa9-9acd500d456c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:25.087847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330ee28-67b0-4497-9462-2b9fa368c060 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:25.095580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-017642c7-d922-4618-9aa9-9acd500d456c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:25.097078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-017642c7-d922-4618-9aa9-9acd500d456c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:25.134308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebabdf-d3f5-478d-9dcb-512317eeea0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:25.134995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebabdf-d3f5-478d-9dcb-512317eeea0b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:25.135400 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebabdf-d3f5-478d-9dcb-512317eeea0b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:25.135682 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebabdf-d3f5-478d-9dcb-512317eeea0b None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:25.136511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebabdf-d3f5-478d-9dcb-512317eeea0b None admin] RBAC: Adding query filter params (type=network) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:25.136819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebabdf-d3f5-478d-9dcb-512317eeea0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:25.138562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebabdf-d3f5-478d-9dcb-512317eeea0b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:25.149376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 366/715] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:25.150601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebabdf-d3f5-478d-9dcb-512317eeea0b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:25.206577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 350/716] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:25.272374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017642c7-d922-4618-9aa9-9acd500d456c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:25.315485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebabdf-d3f5-478d-9dcb-512317eeea0b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:25.378197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09633d9d-19cc-495d-9a23-8335d3246826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:25.379035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09633d9d-19cc-495d-9a23-8335d3246826 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:25.379573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09633d9d-19cc-495d-9a23-8335d3246826 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:25.379850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09633d9d-19cc-495d-9a23-8335d3246826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:25.393534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09633d9d-19cc-495d-9a23-8335d3246826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:25.395221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09633d9d-19cc-495d-9a23-8335d3246826 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:25.404483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-09633d9d-19cc-495d-9a23-8335d3246826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:25.417388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:25.418340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:25.419280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:25.419280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:25.419280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:25.419280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:25.419280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 367/717] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:25.424368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09633d9d-19cc-495d-9a23-8335d3246826 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:25.426501 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017642c7-d922-4618-9aa9-9acd500d456c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:25.457589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:25.478217 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0f21-2c39-432e-8671-4d81e9db409c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:25.479188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0f21-2c39-432e-8671-4d81e9db409c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:25.486556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0f21-2c39-432e-8671-4d81e9db409c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:25.486556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0f21-2c39-432e-8671-4d81e9db409c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:25.504833 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0f21-2c39-432e-8671-4d81e9db409c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-672e0f21-2c39-432e-8671-4d81e9db409c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:25.522325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:25.524309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:25.526459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 351/718] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:25.533193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0f21-2c39-432e-8671-4d81e9db409c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:25.587575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09633d9d-19cc-495d-9a23-8335d3246826 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:25.654372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f3783-2f10-40f1-b8fd-a0287270e094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:25.654686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f3783-2f10-40f1-b8fd-a0287270e094 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:25.654904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f3783-2f10-40f1-b8fd-a0287270e094 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:25.655177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f3783-2f10-40f1-b8fd-a0287270e094 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:25.655919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f3783-2f10-40f1-b8fd-a0287270e094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:25.656221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f3783-2f10-40f1-b8fd-a0287270e094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:25.661969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f3783-2f10-40f1-b8fd-a0287270e094 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:25.698808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 368/719] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:25.710319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f3783-2f10-40f1-b8fd-a0287270e094 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:25.732783 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0f21-2c39-432e-8671-4d81e9db409c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:25.776049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa3c3c-9fea-4f4b-8d9c-ff7ed5bd6b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:25.776730 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3c3c-9fea-4f4b-8d9c-ff7ed5bd6b5d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:25.777014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3c3c-9fea-4f4b-8d9c-ff7ed5bd6b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:25.777319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3c3c-9fea-4f4b-8d9c-ff7ed5bd6b5d None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:25.778459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa3c3c-9fea-4f4b-8d9c-ff7ed5bd6b5d None admin] RBAC: Adding query filter params (name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:25.778767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa3c3c-9fea-4f4b-8d9c-ff7ed5bd6b5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:25.780449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa3c3c-9fea-4f4b-8d9c-ff7ed5bd6b5d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:25.790084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 352/720] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:25.801820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa3c3c-9fea-4f4b-8d9c-ff7ed5bd6b5d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:25.840762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:25.880547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:25.915001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f3783-2f10-40f1-b8fd-a0287270e094 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:25.958834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa3c3c-9fea-4f4b-8d9c-ff7ed5bd6b5d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:25.997187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec2b5e-b472-4127-a078-9f86d097a640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:25.997187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec2b5e-b472-4127-a078-9f86d097a640 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:25.997187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec2b5e-b472-4127-a078-9f86d097a640 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:25.997187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec2b5e-b472-4127-a078-9f86d097a640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.004070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbef0a-ed45-4800-80fa-efaa84804c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:26.005270 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbef0a-ed45-4800-80fa-efaa84804c4a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:26.005771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbef0a-ed45-4800-80fa-efaa84804c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:26.006356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbef0a-ed45-4800-80fa-efaa84804c4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.009202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec2b5e-b472-4127-a078-9f86d097a640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:26.010804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec2b5e-b472-4127-a078-9f86d097a640 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:26.025200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbef0a-ed45-4800-80fa-efaa84804c4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:26.027192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbef0a-ed45-4800-80fa-efaa84804c4a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-bdec2b5e-b472-4127-a078-9f86d097a640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:26.029800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:26.030709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:26.031630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:26.031630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:26.031630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:26.031630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:26.031630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 353/721] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:26.042118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec2b5e-b472-4127-a078-9f86d097a640 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-33fbef0a-ed45-4800-80fa-efaa84804c4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:26.053822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:26.054725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:26.055601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:26.055601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:26.055601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:26.055601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:26.055601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:26.055601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:26.055601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:26.055601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:26.061624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 369/722] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:26.068330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbef0a-ed45-4800-80fa-efaa84804c4a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:26.182279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec2b5e-b472-4127-a078-9f86d097a640 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:26.185485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:26.223826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126691e-0d33-435c-8dd2-d724505fddc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:26.224491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126691e-0d33-435c-8dd2-d724505fddc6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:26.225099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126691e-0d33-435c-8dd2-d724505fddc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:26.225422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126691e-0d33-435c-8dd2-d724505fddc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.226461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126691e-0d33-435c-8dd2-d724505fddc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:26.226771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126691e-0d33-435c-8dd2-d724505fddc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:26.228914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126691e-0d33-435c-8dd2-d724505fddc6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:26.234955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbef0a-ed45-4800-80fa-efaa84804c4a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:26.246921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 354/723] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:26.248675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead0263-c6cf-4388-bf47-340463e19541 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:26.249072 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead0263-c6cf-4388-bf47-340463e19541 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:26.249473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead0263-c6cf-4388-bf47-340463e19541 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.254985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 355/724] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:26.255885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead0263-c6cf-4388-bf47-340463e19541 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:26.256398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:26.277463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacd16c-217f-4002-a1c6-6dc07d17ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:26.278587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacd16c-217f-4002-a1c6-6dc07d17ed81 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:26.278871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacd16c-217f-4002-a1c6-6dc07d17ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:26.279144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacd16c-217f-4002-a1c6-6dc07d17ed81 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.280234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacd16c-217f-4002-a1c6-6dc07d17ed81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:26.280553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacd16c-217f-4002-a1c6-6dc07d17ed81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:26.282489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacd16c-217f-4002-a1c6-6dc07d17ed81 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:26.298837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 370/725] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:26.302550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b387d-8ae6-4aea-acbd-13f121eab279 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:26.302881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b387d-8ae6-4aea-acbd-13f121eab279 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:26.303163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b387d-8ae6-4aea-acbd-13f121eab279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.326815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-645b387d-8ae6-4aea-acbd-13f121eab279 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:26.328316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-645b387d-8ae6-4aea-acbd-13f121eab279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:26.380424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead0263-c6cf-4388-bf47-340463e19541 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:26.422265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 371/726] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:26.423200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b387d-8ae6-4aea-acbd-13f121eab279 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:26.429975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e477c6-c778-4016-8c9e-2ecf4053908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:26.430860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e477c6-c778-4016-8c9e-2ecf4053908e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:26.431251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e477c6-c778-4016-8c9e-2ecf4053908e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:26.431534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e477c6-c778-4016-8c9e-2ecf4053908e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.450852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e477c6-c778-4016-8c9e-2ecf4053908e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:26.452516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e477c6-c778-4016-8c9e-2ecf4053908e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-b2e477c6-c778-4016-8c9e-2ecf4053908e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:26.474968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:26.477196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 356/727] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jul 14 11:24:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:26.479499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e477c6-c778-4016-8c9e-2ecf4053908e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:26.499064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:26.549562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b387d-8ae6-4aea-acbd-13f121eab279 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:26.560475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:26.594413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b910b5-9481-41c8-a363-04d789f3dc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:26.595126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b910b5-9481-41c8-a363-04d789f3dc89 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:26.595351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b910b5-9481-41c8-a363-04d789f3dc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:26.601180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b910b5-9481-41c8-a363-04d789f3dc89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.601180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b910b5-9481-41c8-a363-04d789f3dc89 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=147d764ef17042dbb7ea179c29711588) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:26.601180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b910b5-9481-41c8-a363-04d789f3dc89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:26.601180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b910b5-9481-41c8-a363-04d789f3dc89 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:26.609263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 372/728] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:26 2022] GET /identity/v3/role_assignments?user.id=147d764ef17042dbb7ea179c29711588&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:26.610155 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b910b5-9481-41c8-a363-04d789f3dc89 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:26.618018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e477c6-c778-4016-8c9e-2ecf4053908e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:26.690946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba3131-c4aa-4b07-b7dc-842574113b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:26.691809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba3131-c4aa-4b07-b7dc-842574113b1c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:26.692092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba3131-c4aa-4b07-b7dc-842574113b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:26.692371 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba3131-c4aa-4b07-b7dc-842574113b1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.716749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba3131-c4aa-4b07-b7dc-842574113b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-b2ba3131-c4aa-4b07-b7dc-842574113b1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:26.726419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:26.728606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:26.730650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 357/729] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:26.737270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba3131-c4aa-4b07-b7dc-842574113b1c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:26.837599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b910b5-9481-41c8-a363-04d789f3dc89 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:26.858461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:26.888072 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba3131-c4aa-4b07-b7dc-842574113b1c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:26.891812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ec849-6860-48ce-a348-d43174397c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:26.892423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ec849-6860-48ce-a348-d43174397c0a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:26.892707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ec849-6860-48ce-a348-d43174397c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:26.893028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ec849-6860-48ce-a348-d43174397c0a None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.893930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ec849-6860-48ce-a348-d43174397c0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:26.896382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ec849-6860-48ce-a348-d43174397c0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:26.896382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ec849-6860-48ce-a348-d43174397c0a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:26.933964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 373/730] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jul 14 11:24:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:26.944606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ec849-6860-48ce-a348-d43174397c0a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:26.951790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54616894-427e-46f2-b156-0f8a7fb2e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:26.952786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616894-427e-46f2-b156-0f8a7fb2e0ec None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:26.953315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616894-427e-46f2-b156-0f8a7fb2e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:26.953846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616894-427e-46f2-b156-0f8a7fb2e0ec None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:26.954847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54616894-427e-46f2-b156-0f8a7fb2e0ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:26.955453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54616894-427e-46f2-b156-0f8a7fb2e0ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:26.957090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54616894-427e-46f2-b156-0f8a7fb2e0ec None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:26.966591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 358/731] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:26.972463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54616894-427e-46f2-b156-0f8a7fb2e0ec None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:27.032549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:27.083930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54616894-427e-46f2-b156-0f8a7fb2e0ec None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:27.132230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35e3de-e380-4493-9b36-b91450ada9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:27.133319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35e3de-e380-4493-9b36-b91450ada9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.133607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35e3de-e380-4493-9b36-b91450ada9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.133915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35e3de-e380-4493-9b36-b91450ada9e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.137095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ec849-6860-48ce-a348-d43174397c0a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:27.146804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:27.150855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35e3de-e380-4493-9b36-b91450ada9e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:27.152643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35e3de-e380-4493-9b36-b91450ada9e7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:27.175068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f3e9e-f77b-409f-a60c-e5c164018ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:27.176128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f3e9e-f77b-409f-a60c-e5c164018ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.176610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f3e9e-f77b-409f-a60c-e5c164018ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.177093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f3e9e-f77b-409f-a60c-e5c164018ca5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-9a35e3de-e380-4493-9b36-b91450ada9e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:27.177806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:27.178670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:27.179542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:27.179542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:27.179542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:27.179542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:27.179542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 359/732] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:27.180905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1749b95-a626-4991-9614-4269d7cc65eb None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.181204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1749b95-a626-4991-9614-4269d7cc65eb None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.181483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1749b95-a626-4991-9614-4269d7cc65eb None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.187185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 360/733] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:27.187650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1749b95-a626-4991-9614-4269d7cc65eb None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:27.269013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f3e9e-f77b-409f-a60c-e5c164018ca5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:27.271064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f3e9e-f77b-409f-a60c-e5c164018ca5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:27.307853 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:27.319018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1749b95-a626-4991-9614-4269d7cc65eb None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:27.371396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef33317-61b4-4529-9fdd-354206a8147a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:27.372970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33317-61b4-4529-9fdd-354206a8147a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.373550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33317-61b4-4529-9fdd-354206a8147a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.374263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33317-61b4-4529-9fdd-354206a8147a None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.375460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef33317-61b4-4529-9fdd-354206a8147a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:27.375950 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef33317-61b4-4529-9fdd-354206a8147a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:27.378500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef33317-61b4-4529-9fdd-354206a8147a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:27.390608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 374/734] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:27.390608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a79bd8-4516-4a2b-bab4-522a12c1d8b7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.390608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a79bd8-4516-4a2b-bab4-522a12c1d8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.390608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a79bd8-4516-4a2b-bab4-522a12c1d8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.400834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 361/735] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:27.402631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef33317-61b4-4529-9fdd-354206a8147a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:27.408066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e4a79bd8-4516-4a2b-bab4-522a12c1d8b7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:27.409464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e4a79bd8-4516-4a2b-bab4-522a12c1d8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:27.475846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:27.487802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 375/736] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:27.494213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a79bd8-4516-4a2b-bab4-522a12c1d8b7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:27.534137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef33317-61b4-4529-9fdd-354206a8147a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:27.566146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:27.569946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970ede1-1e8c-4840-9b5a-4227f2b00a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:27.570678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ede1-1e8c-4840-9b5a-4227f2b00a2a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.570958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ede1-1e8c-4840-9b5a-4227f2b00a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.571223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ede1-1e8c-4840-9b5a-4227f2b00a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.582935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970ede1-1e8c-4840-9b5a-4227f2b00a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:27.584571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970ede1-1e8c-4840-9b5a-4227f2b00a2a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-5970ede1-1e8c-4840-9b5a-4227f2b00a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:27.601932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:27.602884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:27.603817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:27.603817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:27.603817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:27.603817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:27.603817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 362/737] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:27.604620 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5f69-1a74-4fe4-9035-8cf0472a240f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.604726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5f69-1a74-4fe4-9035-8cf0472a240f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.605010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5f69-1a74-4fe4-9035-8cf0472a240f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.606440 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 363/738] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:27.610266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c5f69-1a74-4fe4-9035-8cf0472a240f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:27.614814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a79bd8-4516-4a2b-bab4-522a12c1d8b7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:27.655109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d370ce-4055-411d-83cf-749164fae57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:27.655967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d370ce-4055-411d-83cf-749164fae57d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.656298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d370ce-4055-411d-83cf-749164fae57d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.656577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d370ce-4055-411d-83cf-749164fae57d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.664435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d370ce-4055-411d-83cf-749164fae57d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:27.665925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d370ce-4055-411d-83cf-749164fae57d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-83d370ce-4055-411d-83cf-749164fae57d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:27.680487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:27.686216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 376/739] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:27.687264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d370ce-4055-411d-83cf-749164fae57d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:27.726149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c5f69-1a74-4fe4-9035-8cf0472a240f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:27.728571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:27.766644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa52df-0c73-4c92-ae09-48c5d6d1d88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:27.767823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa52df-0c73-4c92-ae09-48c5d6d1d88e None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.768203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa52df-0c73-4c92-ae09-48c5d6d1d88e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.768870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa52df-0c73-4c92-ae09-48c5d6d1d88e None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.770309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa52df-0c73-4c92-ae09-48c5d6d1d88e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:27.771781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa52df-0c73-4c92-ae09-48c5d6d1d88e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:27.790980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d370ce-4055-411d-83cf-749164fae57d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:27.810685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:27.824706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d190f8-8e0e-42a0-b9bd-703842adc8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:27.825649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d190f8-8e0e-42a0-b9bd-703842adc8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.825932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d190f8-8e0e-42a0-b9bd-703842adc8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.826258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d190f8-8e0e-42a0-b9bd-703842adc8f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.826981 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d190f8-8e0e-42a0-b9bd-703842adc8f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:27.827286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d190f8-8e0e-42a0-b9bd-703842adc8f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:27.828642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d190f8-8e0e-42a0-b9bd-703842adc8f9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:27.840258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 377/740] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:27.841483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea21d6-a628-46c0-97a9-c0c3e74980ba None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:27.841772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea21d6-a628-46c0-97a9-c0c3e74980ba None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:27.842048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea21d6-a628-46c0-97a9-c0c3e74980ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:27.858284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-90ea21d6-a628-46c0-97a9-c0c3e74980ba None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:27.859462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-90ea21d6-a628-46c0-97a9-c0c3e74980ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:27.901927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 364/741] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:24:27 2022] POST /identity/v3/services => generated 251 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:27.903652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa52df-0c73-4c92-ae09-48c5d6d1d88e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:27.989808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 378/742] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 11:24:27.991124 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea21d6-a628-46c0-97a9-c0c3e74980ba None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:28.061611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa52df-0c73-4c92-ae09-48c5d6d1d88e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:28.108411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde4974-f0ed-4f0d-80f7-784dc40d3e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:28.109094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde4974-f0ed-4f0d-80f7-784dc40d3e7c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:28.109409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde4974-f0ed-4f0d-80f7-784dc40d3e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:28.109689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde4974-f0ed-4f0d-80f7-784dc40d3e7c None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:28.110542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde4974-f0ed-4f0d-80f7-784dc40d3e7c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:28.110850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde4974-f0ed-4f0d-80f7-784dc40d3e7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:28.112419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde4974-f0ed-4f0d-80f7-784dc40d3e7c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:28.130216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea21d6-a628-46c0-97a9-c0c3e74980ba None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:28.132982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 365/743] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jul 14 11:24:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:28.133989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde4974-f0ed-4f0d-80f7-784dc40d3e7c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:28.141454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:28.162386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:28.173804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a00634-245a-41c4-b36f-bcfd7ee12659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:28.174613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a00634-245a-41c4-b36f-bcfd7ee12659 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:28.174898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a00634-245a-41c4-b36f-bcfd7ee12659 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:28.175173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a00634-245a-41c4-b36f-bcfd7ee12659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:28.182438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a00634-245a-41c4-b36f-bcfd7ee12659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:28.183981 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a00634-245a-41c4-b36f-bcfd7ee12659 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-07a00634-245a-41c4-b36f-bcfd7ee12659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:28.202731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:28.205381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:28.206281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:28.206281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:28.206281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:28.206281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:28.206281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:28.206281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:28.206281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:28.206281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:28.206281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 379/744] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:28.206281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a00634-245a-41c4-b36f-bcfd7ee12659 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:28.274219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde4974-f0ed-4f0d-80f7-784dc40d3e7c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:28.325602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b502f5-8b7a-4e56-bccb-4dfccc72b7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:28.326192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b502f5-8b7a-4e56-bccb-4dfccc72b7ab None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:28.326477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b502f5-8b7a-4e56-bccb-4dfccc72b7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:28.326752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b502f5-8b7a-4e56-bccb-4dfccc72b7ab None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:28.327488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b502f5-8b7a-4e56-bccb-4dfccc72b7ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:28.328874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b502f5-8b7a-4e56-bccb-4dfccc72b7ab None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:28.331782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a00634-245a-41c4-b36f-bcfd7ee12659 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:28.367976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8d015-d21a-44fa-ac84-c875fffd451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:28.368689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8d015-d21a-44fa-ac84-c875fffd451d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:28.368971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8d015-d21a-44fa-ac84-c875fffd451d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:28.369278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8d015-d21a-44fa-ac84-c875fffd451d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:28.407610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:28.417129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 366/745] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:24:28 2022] POST /identity/v3/services => generated 240 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:28.418246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b502f5-8b7a-4e56-bccb-4dfccc72b7ab None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:28.472013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8d015-d21a-44fa-ac84-c875fffd451d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:28.478111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8d015-d21a-44fa-ac84-c875fffd451d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:28.532279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:28.561346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b502f5-8b7a-4e56-bccb-4dfccc72b7ab None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:28.602781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388414df-e4f1-41b3-84c0-a2b3d3d66fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:28.603827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388414df-e4f1-41b3-84c0-a2b3d3d66fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:28.604279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388414df-e4f1-41b3-84c0-a2b3d3d66fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:28.604721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388414df-e4f1-41b3-84c0-a2b3d3d66fb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:28.605763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388414df-e4f1-41b3-84c0-a2b3d3d66fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:28.606314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388414df-e4f1-41b3-84c0-a2b3d3d66fb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:28.608138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388414df-e4f1-41b3-84c0-a2b3d3d66fb0 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:28.623270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 367/746] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:28.628334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388414df-e4f1-41b3-84c0-a2b3d3d66fb0 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:28.652723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 380/747] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:28 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 11:24:28.661107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8d015-d21a-44fa-ac84-c875fffd451d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:28.742648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:28.758177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388414df-e4f1-41b3-84c0-a2b3d3d66fb0 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:28.781506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8d015-d21a-44fa-ac84-c875fffd451d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:28.795181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf28d43-c367-497c-845d-bbe9398218f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:28.799284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf28d43-c367-497c-845d-bbe9398218f2 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:28.799878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf28d43-c367-497c-845d-bbe9398218f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:28.800402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf28d43-c367-497c-845d-bbe9398218f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:28.802626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf28d43-c367-497c-845d-bbe9398218f2 None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=0bc624eba860448eab54b516bd344a46) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:28.803160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf28d43-c367-497c-845d-bbe9398218f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:28.804851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf28d43-c367-497c-845d-bbe9398218f2 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:28.811611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 368/748] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:28 2022] GET /identity/v3/role_assignments?user.id=0bc624eba860448eab54b516bd344a46&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:28.822683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392d70e-df72-4985-8441-3d6827d03aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:28.822992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ce4da-a4c2-4ee9-8c2a-771089aad6c7 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:28.823225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ce4da-a4c2-4ee9-8c2a-771089aad6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:28.823420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ce4da-a4c2-4ee9-8c2a-771089aad6c7 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:28.823576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392d70e-df72-4985-8441-3d6827d03aba None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:28.823875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392d70e-df72-4985-8441-3d6827d03aba None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:28.824156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392d70e-df72-4985-8441-3d6827d03aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:28.824770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 369/749] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:28.830851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1ac80-d317-4539-b08b-70b443d26458 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:28.831171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1ac80-d317-4539-b08b-70b443d26458 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:28.831415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1ac80-d317-4539-b08b-70b443d26458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:28.841221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392d70e-df72-4985-8441-3d6827d03aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:28.841431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:28.841542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392d70e-df72-4985-8441-3d6827d03aba None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:28.874503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-69d1ac80-d317-4539-b08b-70b443d26458 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f392d70e-df72-4985-8441-3d6827d03aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:28.875159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:28.876226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:28.877257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:28.877257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:28.877257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:28.877257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:28.877479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-69d1ac80-d317-4539-b08b-70b443d26458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:28.884252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 381/750] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:28.885396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392d70e-df72-4985-8441-3d6827d03aba None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:29.004496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 370/751] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:24:29.012543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392d70e-df72-4985-8441-3d6827d03aba None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:29.032154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:29.046915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1ac80-d317-4539-b08b-70b443d26458 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:29.052359 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf89a28-d82e-476a-b875-5d8454dd8299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:29.053310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89a28-d82e-476a-b875-5d8454dd8299 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:29.053741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89a28-d82e-476a-b875-5d8454dd8299 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:29.054223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89a28-d82e-476a-b875-5d8454dd8299 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:29.055203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf89a28-d82e-476a-b875-5d8454dd8299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:29.055662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf89a28-d82e-476a-b875-5d8454dd8299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:29.057317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf89a28-d82e-476a-b875-5d8454dd8299 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:29.072111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 382/752] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:29.078035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf89a28-d82e-476a-b875-5d8454dd8299 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:29.183794 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1ac80-d317-4539-b08b-70b443d26458 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:29.187645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf89a28-d82e-476a-b875-5d8454dd8299 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:29.226891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ca93c-c749-41bf-9bb7-9bf0c54d0e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:29.227986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ca93c-c749-41bf-9bb7-9bf0c54d0e14 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:29.228411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ca93c-c749-41bf-9bb7-9bf0c54d0e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:29.228851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ca93c-c749-41bf-9bb7-9bf0c54d0e14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:29.236478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ca93c-c749-41bf-9bb7-9bf0c54d0e14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:29.238262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ca93c-c749-41bf-9bb7-9bf0c54d0e14 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-ca5ca93c-c749-41bf-9bb7-9bf0c54d0e14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:29.257660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:29.258854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:29.259984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:29.259984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:29.259984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:29.259984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:29.259984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:29.259984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:29.259984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:29.259984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:29.262189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 383/753] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:29.266871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:29.269085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ca93c-c749-41bf-9bb7-9bf0c54d0e14 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:29.285968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf91df-15f8-4e56-bfbb-f64a23e036e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:29.288846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf91df-15f8-4e56-bfbb-f64a23e036e2 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:29.288846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf91df-15f8-4e56-bfbb-f64a23e036e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:29.288846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf91df-15f8-4e56-bfbb-f64a23e036e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:29.322667 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf91df-15f8-4e56-bfbb-f64a23e036e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:29.323227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-1aaf91df-15f8-4e56-bfbb-f64a23e036e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:29.338922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:29.339960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:29.340836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 371/754] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:29.350220 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf91df-15f8-4e56-bfbb-f64a23e036e2 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:29.382370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ca93c-c749-41bf-9bb7-9bf0c54d0e14 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:29.419873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d21ef-0cc7-4bd0-a44b-1a05ab40b6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:29.420602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d21ef-0cc7-4bd0-a44b-1a05ab40b6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:29.420884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d21ef-0cc7-4bd0-a44b-1a05ab40b6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:29.421173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d21ef-0cc7-4bd0-a44b-1a05ab40b6c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:29.422004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d21ef-0cc7-4bd0-a44b-1a05ab40b6c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:29.422314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d21ef-0cc7-4bd0-a44b-1a05ab40b6c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:29.423950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d21ef-0cc7-4bd0-a44b-1a05ab40b6c1 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:29.438535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 384/755] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:29.444387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d21ef-0cc7-4bd0-a44b-1a05ab40b6c1 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:29.546770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf91df-15f8-4e56-bfbb-f64a23e036e2 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:29.547665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:29.551674 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d21ef-0cc7-4bd0-a44b-1a05ab40b6c1 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:29.589074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d642ff-60d1-4f1f-ba08-557e83f9249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:29.589764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d642ff-60d1-4f1f-ba08-557e83f9249f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:29.590047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d642ff-60d1-4f1f-ba08-557e83f9249f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:29.590324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d642ff-60d1-4f1f-ba08-557e83f9249f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:29.591166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d642ff-60d1-4f1f-ba08-557e83f9249f None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=42a5ec793f014cefb9d0a0d7258bbb59) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:29.591479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d642ff-60d1-4f1f-ba08-557e83f9249f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:29.593017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d642ff-60d1-4f1f-ba08-557e83f9249f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:29.599700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 385/756] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:29 2022] GET /identity/v3/role_assignments?user.id=42a5ec793f014cefb9d0a0d7258bbb59&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:29.627166 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9470f25-ee99-469b-96e6-2ba8c774c6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:29.627864 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9470f25-ee99-469b-96e6-2ba8c774c6fb None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:29.628143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9470f25-ee99-469b-96e6-2ba8c774c6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:29.628414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9470f25-ee99-469b-96e6-2ba8c774c6fb None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:29.629262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9470f25-ee99-469b-96e6-2ba8c774c6fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:29.629569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9470f25-ee99-469b-96e6-2ba8c774c6fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:29.630983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9470f25-ee99-469b-96e6-2ba8c774c6fb None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:29.646968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 372/757] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:29.652885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d642ff-60d1-4f1f-ba08-557e83f9249f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:29.783739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d642ff-60d1-4f1f-ba08-557e83f9249f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:29.822136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50822fa6-6f82-4833-9568-04b3dc7822a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:29.822947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50822fa6-6f82-4833-9568-04b3dc7822a8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:29.823229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50822fa6-6f82-4833-9568-04b3dc7822a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:29.823488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50822fa6-6f82-4833-9568-04b3dc7822a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:29.835629 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50822fa6-6f82-4833-9568-04b3dc7822a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:29.837120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50822fa6-6f82-4833-9568-04b3dc7822a8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-50822fa6-6f82-4833-9568-04b3dc7822a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:29.856744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:29.858884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:29.860969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:29.860969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:29.860969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:29.860969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:29.860969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 386/758] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:29.868180 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9470f25-ee99-469b-96e6-2ba8c774c6fb None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:29.933084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:29.989340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9470f25-ee99-469b-96e6-2ba8c774c6fb None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:30.043389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7fb2c-6486-4830-9849-6642c758637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:30.043973 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7fb2c-6486-4830-9849-6642c758637f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:30.044252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7fb2c-6486-4830-9849-6642c758637f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:30.044526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7fb2c-6486-4830-9849-6642c758637f None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:30.045281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7fb2c-6486-4830-9849-6642c758637f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:30.045586 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7fb2c-6486-4830-9849-6642c758637f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:30.046963 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7fb2c-6486-4830-9849-6642c758637f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:30.068764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 373/759] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:30.074205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50822fa6-6f82-4833-9568-04b3dc7822a8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:30.096974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:30.188735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50822fa6-6f82-4833-9568-04b3dc7822a8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:30.225332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de1d7b-ac91-4ae9-929b-6942c7c44edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:30.226126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de1d7b-ac91-4ae9-929b-6942c7c44edc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:30.226408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de1d7b-ac91-4ae9-929b-6942c7c44edc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:30.226679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de1d7b-ac91-4ae9-929b-6942c7c44edc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:30.234278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de1d7b-ac91-4ae9-929b-6942c7c44edc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:30.235729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de1d7b-ac91-4ae9-929b-6942c7c44edc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c5de1d7b-ac91-4ae9-929b-6942c7c44edc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:30.250522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:30.251488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:30.252419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 387/760] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jul 14 11:24:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:30.258930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7fb2c-6486-4830-9849-6642c758637f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:30.411511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7fb2c-6486-4830-9849-6642c758637f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:30.434370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:30.464091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed5ff-37b8-4c82-8e27-bf70b57749b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:30.464772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed5ff-37b8-4c82-8e27-bf70b57749b3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:30.465050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed5ff-37b8-4c82-8e27-bf70b57749b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:30.465349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed5ff-37b8-4c82-8e27-bf70b57749b3 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:30.466215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed5ff-37b8-4c82-8e27-bf70b57749b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:30.466522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed5ff-37b8-4c82-8e27-bf70b57749b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:30.467953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed5ff-37b8-4c82-8e27-bf70b57749b3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:30.492784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 374/761] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jul 14 11:24:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:30.498149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de1d7b-ac91-4ae9-929b-6942c7c44edc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:30.530384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d8dc6-2be2-43d3-bdae-4df853c15fc3 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:30.530384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d8dc6-2be2-43d3-bdae-4df853c15fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:30.530678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d8dc6-2be2-43d3-bdae-4df853c15fc3 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:30.532226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 375/762] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:30.539434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c49149-156b-4cab-b951-c561f30b9d29 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:30.539722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c49149-156b-4cab-b951-c561f30b9d29 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:30.539989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c49149-156b-4cab-b951-c561f30b9d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:30.582235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-92c49149-156b-4cab-b951-c561f30b9d29 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:30.583436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-92c49149-156b-4cab-b951-c561f30b9d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:30.593597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:30.615501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de1d7b-ac91-4ae9-929b-6942c7c44edc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:30.652704 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578991d-27ee-4fc0-8967-d6870f60a846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:30.654803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578991d-27ee-4fc0-8967-d6870f60a846 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:30.656450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578991d-27ee-4fc0-8967-d6870f60a846 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:30.657102 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578991d-27ee-4fc0-8967-d6870f60a846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:30.681015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578991d-27ee-4fc0-8967-d6870f60a846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:30.683471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578991d-27ee-4fc0-8967-d6870f60a846 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f578991d-27ee-4fc0-8967-d6870f60a846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:30.714362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:30.715332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:30.716225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:30.716225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:30.716225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:30.716225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:30.726452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 388/763] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:30.729262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 376/764] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:30.731932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578991d-27ee-4fc0-8967-d6870f60a846 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:30.767800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c49149-156b-4cab-b951-c561f30b9d29 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:30.913440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578991d-27ee-4fc0-8967-d6870f60a846 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:30.951706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:30.960719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc128180-40ff-482e-9ffa-4ee2137db5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:30.961538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc128180-40ff-482e-9ffa-4ee2137db5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:30.961858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc128180-40ff-482e-9ffa-4ee2137db5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:30.962185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc128180-40ff-482e-9ffa-4ee2137db5b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:30.963078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc128180-40ff-482e-9ffa-4ee2137db5b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:30.963388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc128180-40ff-482e-9ffa-4ee2137db5b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:30.964764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc128180-40ff-482e-9ffa-4ee2137db5b9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:30.966182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c49149-156b-4cab-b951-c561f30b9d29 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:30.968061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:30.978249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 389/765] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:30.979651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de649c16-6113-42cf-acd9-5054f8683317 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:30.980254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de649c16-6113-42cf-acd9-5054f8683317 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:30.980903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de649c16-6113-42cf-acd9-5054f8683317 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:30.984644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 390/766] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:30.986152 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de649c16-6113-42cf-acd9-5054f8683317 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:31.014992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd97ca-1879-4c27-8eba-80513bf4142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:31.016601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd97ca-1879-4c27-8eba-80513bf4142a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.017131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd97ca-1879-4c27-8eba-80513bf4142a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.017578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd97ca-1879-4c27-8eba-80513bf4142a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.019035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd97ca-1879-4c27-8eba-80513bf4142a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:31.021296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd97ca-1879-4c27-8eba-80513bf4142a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-68fd97ca-1879-4c27-8eba-80513bf4142a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:31.037539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:31.038716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 11:24:31.039601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:31.043605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 377/767] 10.222.0.40 () {56 vars in 1129 bytes} [Thu Jul 14 11:24:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:31.044948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc74f33-ea61-4cf7-94f7-b8c7343e3d93 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.045259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc74f33-ea61-4cf7-94f7-b8c7343e3d93 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.045837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc74f33-ea61-4cf7-94f7-b8c7343e3d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.062829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-0dc74f33-ea61-4cf7-94f7-b8c7343e3d93 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:31.064019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0dc74f33-ea61-4cf7-94f7-b8c7343e3d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:31.119656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de649c16-6113-42cf-acd9-5054f8683317 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:31.135728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 378/768] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:31.146017 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc74f33-ea61-4cf7-94f7-b8c7343e3d93 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:31.167134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1885e6-d622-4a26-9ca0-e95d957b3404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:31.168212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1885e6-d622-4a26-9ca0-e95d957b3404 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.168634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1885e6-d622-4a26-9ca0-e95d957b3404 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.169045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1885e6-d622-4a26-9ca0-e95d957b3404 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.179108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1885e6-d622-4a26-9ca0-e95d957b3404 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:31.181333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1885e6-d622-4a26-9ca0-e95d957b3404 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-cf1885e6-d622-4a26-9ca0-e95d957b3404 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:31.201597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:31.202768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:31.203872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:31.203872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:31.203872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:31.203872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:31.203872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:31.203872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:31.203872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:31.203872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:31.206897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 391/769] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:31.211637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1885e6-d622-4a26-9ca0-e95d957b3404 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:31.262181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:31.280411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:31.291581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc74f33-ea61-4cf7-94f7-b8c7343e3d93 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:31.329578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1885e6-d622-4a26-9ca0-e95d957b3404 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:31.336753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8d7ff-7318-4cc7-9070-c14284e59e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:31.337644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8d7ff-7318-4cc7-9070-c14284e59e3a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.338485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8d7ff-7318-4cc7-9070-c14284e59e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.338917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8d7ff-7318-4cc7-9070-c14284e59e3a None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.339797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8d7ff-7318-4cc7-9070-c14284e59e3a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:31.340263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8d7ff-7318-4cc7-9070-c14284e59e3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:31.342483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8d7ff-7318-4cc7-9070-c14284e59e3a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:31.347853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 379/770] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jul 14 11:24:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:31.348827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8d7ff-7318-4cc7-9070-c14284e59e3a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:31.376647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4819f87-a016-49a8-ba75-92ec0680bb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:31.377725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819f87-a016-49a8-ba75-92ec0680bb1c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.378420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819f87-a016-49a8-ba75-92ec0680bb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.379212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819f87-a016-49a8-ba75-92ec0680bb1c None admin] PATH_INFO: `/v3/services/network` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.380163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4819f87-a016-49a8-ba75-92ec0680bb1c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:31.382504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4819f87-a016-49a8-ba75-92ec0680bb1c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-d4819f87-a016-49a8-ba75-92ec0680bb1c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:31.401460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:31.402493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:31.403814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 392/771] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:31 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:31.406327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6b446-62ec-4cde-ac00-bb9544623720 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.406668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6b446-62ec-4cde-ac00-bb9544623720 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.406992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6b446-62ec-4cde-ac00-bb9544623720 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.408538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 393/772] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:31.410283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6b446-62ec-4cde-ac00-bb9544623720 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:31.462783 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8d7ff-7318-4cc7-9070-c14284e59e3a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:31.501510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394f5a6-02f0-4e9f-bc15-500131ee03b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:31.502167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394f5a6-02f0-4e9f-bc15-500131ee03b1 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.502450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394f5a6-02f0-4e9f-bc15-500131ee03b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.502723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394f5a6-02f0-4e9f-bc15-500131ee03b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.503511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394f5a6-02f0-4e9f-bc15-500131ee03b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:31.504355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394f5a6-02f0-4e9f-bc15-500131ee03b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:31.505824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394f5a6-02f0-4e9f-bc15-500131ee03b1 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:31.518199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 380/773] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:31.518998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394f5a6-02f0-4e9f-bc15-500131ee03b1 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:31.534266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6b446-62ec-4cde-ac00-bb9544623720 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:31.544984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:31.545375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:31.568971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c52437-5d1f-4157-8205-299c2bec4095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:31.569971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52437-5d1f-4157-8205-299c2bec4095 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.570431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52437-5d1f-4157-8205-299c2bec4095 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.570707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52437-5d1f-4157-8205-299c2bec4095 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.571424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c52437-5d1f-4157-8205-299c2bec4095 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:31.571732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c52437-5d1f-4157-8205-299c2bec4095 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:31.573143 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c52437-5d1f-4157-8205-299c2bec4095 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:31.578209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 394/774] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jul 14 11:24:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:31.584739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce8fc9-c52c-4c13-9a8b-fa3d35bcc618 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.585028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce8fc9-c52c-4c13-9a8b-fa3d35bcc618 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.585337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce8fc9-c52c-4c13-9a8b-fa3d35bcc618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.603188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-43ce8fc9-c52c-4c13-9a8b-fa3d35bcc618 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:31.607950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-43ce8fc9-c52c-4c13-9a8b-fa3d35bcc618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:31.626398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394f5a6-02f0-4e9f-bc15-500131ee03b1 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:31.668968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf2dc2-1cce-4f60-a384-9fbd6d238238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:31.669664 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf2dc2-1cce-4f60-a384-9fbd6d238238 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.669954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf2dc2-1cce-4f60-a384-9fbd6d238238 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.670238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf2dc2-1cce-4f60-a384-9fbd6d238238 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.670960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf2dc2-1cce-4f60-a384-9fbd6d238238 None admin] RBAC: Adding query filter params (name=network) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:31.671269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf2dc2-1cce-4f60-a384-9fbd6d238238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:31.672652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf2dc2-1cce-4f60-a384-9fbd6d238238 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:31.677951 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 381/775] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:31.678709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf2dc2-1cce-4f60-a384-9fbd6d238238 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:31.684896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 395/776] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:31.686593 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce8fc9-c52c-4c13-9a8b-fa3d35bcc618 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:31.773567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:31.790950 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf2dc2-1cce-4f60-a384-9fbd6d238238 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:31.805548 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce8fc9-c52c-4c13-9a8b-fa3d35bcc618 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:31.808542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:31.829458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ccf9a-4af9-4536-84d4-6dab56cf65ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:31.830355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ccf9a-4af9-4536-84d4-6dab56cf65ca None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.830887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ccf9a-4af9-4536-84d4-6dab56cf65ca None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.830887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ccf9a-4af9-4536-84d4-6dab56cf65ca None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/147d764ef17042dbb7ea179c29711588/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.833552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ccf9a-4af9-4536-84d4-6dab56cf65ca None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=147d764ef17042dbb7ea179c29711588, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:31.835003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ccf9a-4af9-4536-84d4-6dab56cf65ca None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:31.841959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b739e-1c55-4c56-844d-83f8e74ce4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:31.842652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b739e-1c55-4c56-844d-83f8e74ce4cb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:31.843004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b739e-1c55-4c56-844d-83f8e74ce4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:31.843285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b739e-1c55-4c56-844d-83f8e74ce4cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:31.844079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b739e-1c55-4c56-844d-83f8e74ce4cb None admin] RBAC: Adding query filter params (interface=public, service_id=0cef86f7a7734d46b479b38b172c2f6a, region_id=RegionOne) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:31.844391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b739e-1c55-4c56-844d-83f8e74ce4cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:31.845801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b739e-1c55-4c56-844d-83f8e74ce4cb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:31.854042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 396/777] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jul 14 11:24:31 2022] GET /identity/v3/endpoints?service_id=0cef86f7a7734d46b479b38b172c2f6a&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:31.854911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b739e-1c55-4c56-844d-83f8e74ce4cb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:31.933564 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 382/778] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:24:31 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/147d764ef17042dbb7ea179c29711588/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:31.934718 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ccf9a-4af9-4536-84d4-6dab56cf65ca None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:31.966513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b739e-1c55-4c56-844d-83f8e74ce4cb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:32.009028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a4218-adc4-452b-9fea-981ccfa4c3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:32.009641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4218-adc4-452b-9fea-981ccfa4c3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.009926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4218-adc4-452b-9fea-981ccfa4c3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.010247 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4218-adc4-452b-9fea-981ccfa4c3c4 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.011054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a4218-adc4-452b-9fea-981ccfa4c3c4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:32.011362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a4218-adc4-452b-9fea-981ccfa4c3c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:32.012824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a4218-adc4-452b-9fea-981ccfa4c3c4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:32.024364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 397/779] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:32.025351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a4218-adc4-452b-9fea-981ccfa4c3c4 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:32.061551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:32.092218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ccf9a-4af9-4536-84d4-6dab56cf65ca None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:32.134630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d74059-0543-4dba-bd92-487c5828a514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:32.135936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d74059-0543-4dba-bd92-487c5828a514 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.136551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d74059-0543-4dba-bd92-487c5828a514 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.136837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d74059-0543-4dba-bd92-487c5828a514 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.143518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a4218-adc4-452b-9fea-981ccfa4c3c4 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:32.150168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d74059-0543-4dba-bd92-487c5828a514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-c9d74059-0543-4dba-bd92-487c5828a514 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:32.157997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:32.158966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:32.159875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 383/780] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:32.160579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d74059-0543-4dba-bd92-487c5828a514 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:32.188061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:32.190410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd8301-15a4-4c5b-9cab-98b55c427a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:32.191846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd8301-15a4-4c5b-9cab-98b55c427a2d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.192132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd8301-15a4-4c5b-9cab-98b55c427a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.192409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd8301-15a4-4c5b-9cab-98b55c427a2d None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.193110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd8301-15a4-4c5b-9cab-98b55c427a2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:32.194557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd8301-15a4-4c5b-9cab-98b55c427a2d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:32.203564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 398/781] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jul 14 11:24:32 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:32.204423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd8301-15a4-4c5b-9cab-98b55c427a2d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:32.289775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d74059-0543-4dba-bd92-487c5828a514 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:32.318762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:32.325720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd8301-15a4-4c5b-9cab-98b55c427a2d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:32.342775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a36a-89a5-4a0e-86fe-8bb983d95c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:32.343762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a36a-89a5-4a0e-86fe-8bb983d95c8e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.344053 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a36a-89a5-4a0e-86fe-8bb983d95c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.344331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a36a-89a5-4a0e-86fe-8bb983d95c8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.345343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a36a-89a5-4a0e-86fe-8bb983d95c8e None admin] RBAC: Adding query filter params (interface=public, service_id=4ab4544c2b304676b56f7dc82825f8b5, region_id=RegionOne) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:32.345761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a36a-89a5-4a0e-86fe-8bb983d95c8e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:32.347582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a36a-89a5-4a0e-86fe-8bb983d95c8e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:32.358622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 384/782] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jul 14 11:24:32 2022] GET /identity/v3/endpoints?service_id=4ab4544c2b304676b56f7dc82825f8b5&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:32.360916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02bde-d2df-4e88-a2f5-87a5463903ca None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.361228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02bde-d2df-4e88-a2f5-87a5463903ca None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.361574 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02bde-d2df-4e88-a2f5-87a5463903ca None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.363842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 385/783] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:32.367416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c02bde-d2df-4e88-a2f5-87a5463903ca None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:32.372942 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aaab69-c609-42a4-b9f3-fb0708ca4deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:32.373659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaab69-c609-42a4-b9f3-fb0708ca4deb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.373941 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaab69-c609-42a4-b9f3-fb0708ca4deb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.374215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaab69-c609-42a4-b9f3-fb0708ca4deb None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.374977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aaab69-c609-42a4-b9f3-fb0708ca4deb None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:32.375278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aaab69-c609-42a4-b9f3-fb0708ca4deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:32.376736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aaab69-c609-42a4-b9f3-fb0708ca4deb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:32.388116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 399/784] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:32.389349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd202a31-51b4-41d5-8991-5a40a4ca5a58 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.389679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd202a31-51b4-41d5-8991-5a40a4ca5a58 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.389975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd202a31-51b4-41d5-8991-5a40a4ca5a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.408519 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fd202a31-51b4-41d5-8991-5a40a4ca5a58 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:32.409840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fd202a31-51b4-41d5-8991-5a40a4ca5a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:32.454245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:32.491299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c02bde-d2df-4e88-a2f5-87a5463903ca None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:32.492376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 400/785] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:32.494188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd202a31-51b4-41d5-8991-5a40a4ca5a58 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:32.529334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693e4f5-8414-40ac-99c6-ba01d0963af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:32.530002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693e4f5-8414-40ac-99c6-ba01d0963af0 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.530288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693e4f5-8414-40ac-99c6-ba01d0963af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.530564 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693e4f5-8414-40ac-99c6-ba01d0963af0 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.531258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693e4f5-8414-40ac-99c6-ba01d0963af0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:32.532672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693e4f5-8414-40ac-99c6-ba01d0963af0 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:32.540041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 386/786] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jul 14 11:24:32 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:32.540936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693e4f5-8414-40ac-99c6-ba01d0963af0 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:32.663841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693e4f5-8414-40ac-99c6-ba01d0963af0 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:32.707390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:32.711452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5fc4f-3567-4492-8057-8935066b3172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:32.712389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5fc4f-3567-4492-8057-8935066b3172 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.712752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5fc4f-3567-4492-8057-8935066b3172 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.713105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5fc4f-3567-4492-8057-8935066b3172 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.727420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5fc4f-3567-4492-8057-8935066b3172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:32.728086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-a7f5fc4f-3567-4492-8057-8935066b3172 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:32.736149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:32.737119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:32.738045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 387/787] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:32.745103 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5fc4f-3567-4492-8057-8935066b3172 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:32.749364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd202a31-51b4-41d5-8991-5a40a4ca5a58 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:32.788007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d42f21-5179-4d11-b310-675f5119a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:32.789525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d42f21-5179-4d11-b310-675f5119a77f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.790110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d42f21-5179-4d11-b310-675f5119a77f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.790401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d42f21-5179-4d11-b310-675f5119a77f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.808058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d42f21-5179-4d11-b310-675f5119a77f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:32.809828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d42f21-5179-4d11-b310-675f5119a77f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-e8d42f21-5179-4d11-b310-675f5119a77f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:32.828749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:32.829905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:32.830918 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:32.830918 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:32.830918 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:32.830918 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:32.833078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 401/788] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:32.838323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d42f21-5179-4d11-b310-675f5119a77f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:32.854275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5fc4f-3567-4492-8057-8935066b3172 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:32.898623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2ea9f-b858-4613-87be-a471a23866ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:32.899311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2ea9f-b858-4613-87be-a471a23866ad None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:32.899594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2ea9f-b858-4613-87be-a471a23866ad None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:32.899864 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2ea9f-b858-4613-87be-a471a23866ad None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:32.900635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2ea9f-b858-4613-87be-a471a23866ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:32.900993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2ea9f-b858-4613-87be-a471a23866ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:32.902541 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2ea9f-b858-4613-87be-a471a23866ad None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:32.917208 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 388/789] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:32.919193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2ea9f-b858-4613-87be-a471a23866ad None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:32.973332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d42f21-5179-4d11-b310-675f5119a77f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:33.020859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a8ce3-c405-472e-8746-1d9ac2e2b2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:33.021924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a8ce3-c405-472e-8746-1d9ac2e2b2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:33.022794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a8ce3-c405-472e-8746-1d9ac2e2b2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:33.023286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a8ce3-c405-472e-8746-1d9ac2e2b2f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:33.024224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a8ce3-c405-472e-8746-1d9ac2e2b2f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:33.024736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a8ce3-c405-472e-8746-1d9ac2e2b2f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:33.026519 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a8ce3-c405-472e-8746-1d9ac2e2b2f0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:33.036839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:33.053516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 402/790] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:33.053516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a8ce3-c405-472e-8746-1d9ac2e2b2f0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:33.061514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:33.122257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2ea9f-b858-4613-87be-a471a23866ad None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:33.176001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c87b0d-de09-4511-99a1-f323c611c325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:33.176857 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c87b0d-de09-4511-99a1-f323c611c325 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:33.177182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c87b0d-de09-4511-99a1-f323c611c325 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:33.177519 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c87b0d-de09-4511-99a1-f323c611c325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:33.206258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c87b0d-de09-4511-99a1-f323c611c325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:33.208290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c87b0d-de09-4511-99a1-f323c611c325 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-06c87b0d-de09-4511-99a1-f323c611c325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:33.246215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:33.247304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:33.248267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:33.248267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:33.248267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:33.248267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:33.248267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 389/791] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:33.256673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c87b0d-de09-4511-99a1-f323c611c325 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:33.256874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a8ce3-c405-472e-8746-1d9ac2e2b2f0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:33.326035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578993b-eb6f-46b6-b26e-1d7de5db9a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:33.326780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578993b-eb6f-46b6-b26e-1d7de5db9a0b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:33.327061 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578993b-eb6f-46b6-b26e-1d7de5db9a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:33.327330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578993b-eb6f-46b6-b26e-1d7de5db9a0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:33.346241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578993b-eb6f-46b6-b26e-1d7de5db9a0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:33.347717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578993b-eb6f-46b6-b26e-1d7de5db9a0b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-e578993b-eb6f-46b6-b26e-1d7de5db9a0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:33.380706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:33.381781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:33.382725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 403/792] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jul 14 11:24:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:33.383304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c87b0d-de09-4511-99a1-f323c611c325 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:33.387761 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578993b-eb6f-46b6-b26e-1d7de5db9a0b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:33.390984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:33.420273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b902ea6-f94b-45d7-a0e7-fff6e29ffb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:33.420947 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b902ea6-f94b-45d7-a0e7-fff6e29ffb96 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:33.421271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b902ea6-f94b-45d7-a0e7-fff6e29ffb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:33.421555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b902ea6-f94b-45d7-a0e7-fff6e29ffb96 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:33.422509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b902ea6-f94b-45d7-a0e7-fff6e29ffb96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:33.422874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b902ea6-f94b-45d7-a0e7-fff6e29ffb96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:33.424266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b902ea6-f94b-45d7-a0e7-fff6e29ffb96 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:33.437443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 390/793] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:33.444775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b902ea6-f94b-45d7-a0e7-fff6e29ffb96 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:33.514226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:33.558693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578993b-eb6f-46b6-b26e-1d7de5db9a0b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:33.594993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0715ac-5f14-4c79-b9a3-e0974e1b0eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:33.596191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0715ac-5f14-4c79-b9a3-e0974e1b0eba None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:33.596935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0715ac-5f14-4c79-b9a3-e0974e1b0eba None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:33.598044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0715ac-5f14-4c79-b9a3-e0974e1b0eba None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:33.599175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0715ac-5f14-4c79-b9a3-e0974e1b0eba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:33.599786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0715ac-5f14-4c79-b9a3-e0974e1b0eba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:33.601609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0715ac-5f14-4c79-b9a3-e0974e1b0eba None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:33.603395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b902ea6-f94b-45d7-a0e7-fff6e29ffb96 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:33.622430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 404/794] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jul 14 11:24:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:33.628414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0715ac-5f14-4c79-b9a3-e0974e1b0eba None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:33.662300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebfea7-14fe-45cb-bb80-4ba50193e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:33.663262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebfea7-14fe-45cb-bb80-4ba50193e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:33.663599 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebfea7-14fe-45cb-bb80-4ba50193e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:33.663881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebfea7-14fe-45cb-bb80-4ba50193e1b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:33.675149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebfea7-14fe-45cb-bb80-4ba50193e1b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:33.681954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebfea7-14fe-45cb-bb80-4ba50193e1b0 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-78ebfea7-14fe-45cb-bb80-4ba50193e1b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:33.703299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:33.704288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:33.705256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:33.705256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:33.705256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:33.705256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:33.705256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:33.705256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:33.705256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:33.705256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:33.705256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:33.705256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:33.713592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 391/795] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:33.722103 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebfea7-14fe-45cb-bb80-4ba50193e1b0 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:33.739703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0715ac-5f14-4c79-b9a3-e0974e1b0eba None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:33.762439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:33.776256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034405a3-5e1d-4049-b215-1817c1217f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:33.777508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034405a3-5e1d-4049-b215-1817c1217f9b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:33.778080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034405a3-5e1d-4049-b215-1817c1217f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:33.778543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034405a3-5e1d-4049-b215-1817c1217f9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:33.790428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034405a3-5e1d-4049-b215-1817c1217f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:33.792710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034405a3-5e1d-4049-b215-1817c1217f9b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-034405a3-5e1d-4049-b215-1817c1217f9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:33.813263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:33.814154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:33.815042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:33.815042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:33.815042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:33.815042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:33.817343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:33.818622 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 405/796] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:33.824528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034405a3-5e1d-4049-b215-1817c1217f9b None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:33.979107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebfea7-14fe-45cb-bb80-4ba50193e1b0 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:34.020017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034405a3-5e1d-4049-b215-1817c1217f9b None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:34.044804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ac20f-50fd-4b1a-904d-4495cf129f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:34.045513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ac20f-50fd-4b1a-904d-4495cf129f24 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.045800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ac20f-50fd-4b1a-904d-4495cf129f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.046074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ac20f-50fd-4b1a-904d-4495cf129f24 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.046936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ac20f-50fd-4b1a-904d-4495cf129f24 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:34.047241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ac20f-50fd-4b1a-904d-4495cf129f24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:34.051310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ac20f-50fd-4b1a-904d-4495cf129f24 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:34.092734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 392/797] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jul 14 11:24:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:34.094171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ac20f-50fd-4b1a-904d-4495cf129f24 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:34.109907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6496e-e6bb-4c56-9c02-6ae623da4b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:34.110594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6496e-e6bb-4c56-9c02-6ae623da4b94 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.110877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6496e-e6bb-4c56-9c02-6ae623da4b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.111159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6496e-e6bb-4c56-9c02-6ae623da4b94 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.111898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6496e-e6bb-4c56-9c02-6ae623da4b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:34.112202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6496e-e6bb-4c56-9c02-6ae623da4b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:34.113641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6496e-e6bb-4c56-9c02-6ae623da4b94 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:34.150269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 406/798] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:34.156353 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6496e-e6bb-4c56-9c02-6ae623da4b94 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:34.225535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:34.249274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:34.263290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ac20f-50fd-4b1a-904d-4495cf129f24 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:34.348569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16057e2c-0f96-4d35-8998-f1484771636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:34.349447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16057e2c-0f96-4d35-8998-f1484771636c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.349728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16057e2c-0f96-4d35-8998-f1484771636c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.349997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16057e2c-0f96-4d35-8998-f1484771636c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.372429 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16057e2c-0f96-4d35-8998-f1484771636c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:34.374888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16057e2c-0f96-4d35-8998-f1484771636c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:34.379777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6496e-e6bb-4c56-9c02-6ae623da4b94 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-16057e2c-0f96-4d35-8998-f1484771636c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:34.410375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:34.411357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:34.412284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:34.412284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:34.412284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:34.412284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:34.412284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 393/799] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:34.413264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37c509-73db-4bc7-9261-c7528e2a2171 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.416984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37c509-73db-4bc7-9261-c7528e2a2171 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.416984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37c509-73db-4bc7-9261-c7528e2a2171 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.419172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 394/800] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:34.420569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45d4b2-6503-4c99-a08f-ea138f10ef4f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.420959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45d4b2-6503-4c99-a08f-ea138f10ef4f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.421260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45d4b2-6503-4c99-a08f-ea138f10ef4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.433914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04376140-1c8c-436e-a818-0ad65b51aa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:34.434780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04376140-1c8c-436e-a818-0ad65b51aa02 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.435339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04376140-1c8c-436e-a818-0ad65b51aa02 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.435618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04376140-1c8c-436e-a818-0ad65b51aa02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.439071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-6a45d4b2-6503-4c99-a08f-ea138f10ef4f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:34.440337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6a45d4b2-6503-4c99-a08f-ea138f10ef4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:34.443095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04376140-1c8c-436e-a818-0ad65b51aa02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:34.445443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04376140-1c8c-436e-a818-0ad65b51aa02 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-04376140-1c8c-436e-a818-0ad65b51aa02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:34.463320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:34.464938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:34.466221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:34.466221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:34.466221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:34.466221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:34.466221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:34.466221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:34.466221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:34.466221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:34.466221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 407/801] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:34.466221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04376140-1c8c-436e-a818-0ad65b51aa02 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:34.519068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 395/802] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:34.527188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45d4b2-6503-4c99-a08f-ea138f10ef4f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:34.573956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:34.585488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04376140-1c8c-436e-a818-0ad65b51aa02 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:34.607651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:34.628876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f132c38-aced-423d-9e63-23113c492714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:34.629577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f132c38-aced-423d-9e63-23113c492714 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.629861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f132c38-aced-423d-9e63-23113c492714 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.630138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f132c38-aced-423d-9e63-23113c492714 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.630867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f132c38-aced-423d-9e63-23113c492714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:34.631231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f132c38-aced-423d-9e63-23113c492714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:34.632612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f132c38-aced-423d-9e63-23113c492714 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:34.643742 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45d4b2-6503-4c99-a08f-ea138f10ef4f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:34.652947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 408/803] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:34.652947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c75cdd-350d-431f-a262-3204045e28c6 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.652947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c75cdd-350d-431f-a262-3204045e28c6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.652947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c75cdd-350d-431f-a262-3204045e28c6 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.652947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 409/804] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:34.653461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c75cdd-350d-431f-a262-3204045e28c6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:34.695057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d4039-a5c2-4dd0-aa97-6859daa938fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:34.695723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d4039-a5c2-4dd0-aa97-6859daa938fb None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.696127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d4039-a5c2-4dd0-aa97-6859daa938fb None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.696407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d4039-a5c2-4dd0-aa97-6859daa938fb None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.697225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d4039-a5c2-4dd0-aa97-6859daa938fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:34.697539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d4039-a5c2-4dd0-aa97-6859daa938fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:34.699109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d4039-a5c2-4dd0-aa97-6859daa938fb None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:34.717202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 396/805] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:34.717202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370066ac-26a0-4bde-b039-07a72afb6efb None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.717202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370066ac-26a0-4bde-b039-07a72afb6efb None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.717202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370066ac-26a0-4bde-b039-07a72afb6efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.732566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-370066ac-26a0-4bde-b039-07a72afb6efb None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:34.733838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-370066ac-26a0-4bde-b039-07a72afb6efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:34.766433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c75cdd-350d-431f-a262-3204045e28c6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:34.801622 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03182475-e013-456c-830b-c7072ddcae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:34.802426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03182475-e013-456c-830b-c7072ddcae36 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.802760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03182475-e013-456c-830b-c7072ddcae36 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.803037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03182475-e013-456c-830b-c7072ddcae36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.813338 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 397/806] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:34.815612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03182475-e013-456c-830b-c7072ddcae36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:34.818090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370066ac-26a0-4bde-b039-07a72afb6efb None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-03182475-e013-456c-830b-c7072ddcae36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:34.823859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:34.824837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:34.826063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:34.830328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 410/807] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:34.831177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03182475-e013-456c-830b-c7072ddcae36 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:34.844399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:34.890197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:34.940397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370066ac-26a0-4bde-b039-07a72afb6efb None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:34.981909 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebd754-7ac3-4981-b621-8d028b35dda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:34.983172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebd754-7ac3-4981-b621-8d028b35dda7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.983616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebd754-7ac3-4981-b621-8d028b35dda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.983954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebd754-7ac3-4981-b621-8d028b35dda7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.992710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebd754-7ac3-4981-b621-8d028b35dda7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:34.994630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebd754-7ac3-4981-b621-8d028b35dda7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-c0ebd754-7ac3-4981-b621-8d028b35dda7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:35.009687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:35.010774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:35.011687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:35.011687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:35.011687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:35.011687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:35.011687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:35.011687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:35.011687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:35.011687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:35.011687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 398/808] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:35.012175 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebd754-7ac3-4981-b621-8d028b35dda7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:35.064273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03182475-e013-456c-830b-c7072ddcae36 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:35.098420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff7aa5-8fce-4804-a732-42db54556e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:35.099295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff7aa5-8fce-4804-a732-42db54556e80 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.099573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff7aa5-8fce-4804-a732-42db54556e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.099839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff7aa5-8fce-4804-a732-42db54556e80 None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/0bc624eba860448eab54b516bd344a46/roles/60130d2ba383400091dc5f253d40a43a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.114355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebd754-7ac3-4981-b621-8d028b35dda7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:35.114516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff7aa5-8fce-4804-a732-42db54556e80 None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=0bc624eba860448eab54b516bd344a46, role_id=60130d2ba383400091dc5f253d40a43a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:35.115829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff7aa5-8fce-4804-a732-42db54556e80 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:35.131045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:35.148775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d19be4-cfba-40d3-ab65-f57e461bfae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:35.149465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19be4-cfba-40d3-ab65-f57e461bfae2 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.149747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19be4-cfba-40d3-ab65-f57e461bfae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.150019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19be4-cfba-40d3-ab65-f57e461bfae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.150765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d19be4-cfba-40d3-ab65-f57e461bfae2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:35.151071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d19be4-cfba-40d3-ab65-f57e461bfae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:35.152424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d19be4-cfba-40d3-ab65-f57e461bfae2 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:35.160276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 399/809] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:35.161047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d19be4-cfba-40d3-ab65-f57e461bfae2 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:35.196491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 411/810] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:24:34 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/0bc624eba860448eab54b516bd344a46/roles/60130d2ba383400091dc5f253d40a43a => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:35.197421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff7aa5-8fce-4804-a732-42db54556e80 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:35.268521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:35.271819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d19be4-cfba-40d3-ab65-f57e461bfae2 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:35.312820 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff7aa5-8fce-4804-a732-42db54556e80 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:35.316668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feed096-29d7-41e9-9fd9-43895166eccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:35.317554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed096-29d7-41e9-9fd9-43895166eccc None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.317844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed096-29d7-41e9-9fd9-43895166eccc None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.318124 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed096-29d7-41e9-9fd9-43895166eccc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.318809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feed096-29d7-41e9-9fd9-43895166eccc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:35.320304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feed096-29d7-41e9-9fd9-43895166eccc None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-4feed096-29d7-41e9-9fd9-43895166eccc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:35.339634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:35.340792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:35.341911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 400/811] 10.222.0.40 () {56 vars in 1129 bytes} [Thu Jul 14 11:24:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:35.343071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bc4be-7c91-4d6e-8da6-5e2b9b0717ba None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.343364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bc4be-7c91-4d6e-8da6-5e2b9b0717ba None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.343645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bc4be-7c91-4d6e-8da6-5e2b9b0717ba None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.345129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 401/812] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:35.348493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bc4be-7c91-4d6e-8da6-5e2b9b0717ba None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:35.367488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54fe4-0b18-409d-a837-485fa68a228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:35.368244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54fe4-0b18-409d-a837-485fa68a228d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.369200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54fe4-0b18-409d-a837-485fa68a228d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.369200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54fe4-0b18-409d-a837-485fa68a228d None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.369832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d54fe4-0b18-409d-a837-485fa68a228d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:35.370141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d54fe4-0b18-409d-a837-485fa68a228d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:35.371577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d54fe4-0b18-409d-a837-485fa68a228d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:35.385714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 412/813] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:35.386559 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54fe4-0b18-409d-a837-485fa68a228d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:35.403785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:35.463291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bc4be-7c91-4d6e-8da6-5e2b9b0717ba None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:35.496212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54fe4-0b18-409d-a837-485fa68a228d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:35.497740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc193c0-e987-488f-9c56-62b3cf895852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:35.498534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc193c0-e987-488f-9c56-62b3cf895852 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.498821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc193c0-e987-488f-9c56-62b3cf895852 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.499095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc193c0-e987-488f-9c56-62b3cf895852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.510856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc193c0-e987-488f-9c56-62b3cf895852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:35.512356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc193c0-e987-488f-9c56-62b3cf895852 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:35.518890 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-2fc193c0-e987-488f-9c56-62b3cf895852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:35.530792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:35.536509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:35.537446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:35.537446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:35.537446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:35.537446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:35.537446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 402/814] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:35.537446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db861684-d244-4058-b448-17987511f89f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.537446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db861684-d244-4058-b448-17987511f89f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.537446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db861684-d244-4058-b448-17987511f89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.539321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6e97-e384-4a10-86d1-b2daf09aa880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:35.540001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6e97-e384-4a10-86d1-b2daf09aa880 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.540272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6e97-e384-4a10-86d1-b2daf09aa880 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.540601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6e97-e384-4a10-86d1-b2daf09aa880 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.541275 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6e97-e384-4a10-86d1-b2daf09aa880 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:35.541598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6e97-e384-4a10-86d1-b2daf09aa880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:35.542953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6e97-e384-4a10-86d1-b2daf09aa880 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:35.552675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 413/815] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jul 14 11:24:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:35.552675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6e97-e384-4a10-86d1-b2daf09aa880 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:35.562789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-db861684-d244-4058-b448-17987511f89f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:35.564163 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-db861684-d244-4058-b448-17987511f89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:35.639159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 403/816] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:35.640425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db861684-d244-4058-b448-17987511f89f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:35.666170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6e97-e384-4a10-86d1-b2daf09aa880 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:35.682336 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:35.703736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e440d25-fd79-4e24-b2dd-67ef760c0e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:35.704693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e440d25-fd79-4e24-b2dd-67ef760c0e62 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.705001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e440d25-fd79-4e24-b2dd-67ef760c0e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.705266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e440d25-fd79-4e24-b2dd-67ef760c0e62 None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/42a5ec793f014cefb9d0a0d7258bbb59/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.707295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e440d25-fd79-4e24-b2dd-67ef760c0e62 None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=42a5ec793f014cefb9d0a0d7258bbb59, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:35.708712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e440d25-fd79-4e24-b2dd-67ef760c0e62 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:35.774983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db861684-d244-4058-b448-17987511f89f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:35.813188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 414/817] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:24:35 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/42a5ec793f014cefb9d0a0d7258bbb59/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:35.813188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e440d25-fd79-4e24-b2dd-67ef760c0e62 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:35.824859 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc19653-eb73-44ed-aa7c-f33c8ca3461e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:35.826419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc19653-eb73-44ed-aa7c-f33c8ca3461e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.827005 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc19653-eb73-44ed-aa7c-f33c8ca3461e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.827345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc19653-eb73-44ed-aa7c-f33c8ca3461e None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.828232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc19653-eb73-44ed-aa7c-f33c8ca3461e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:35.828737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc19653-eb73-44ed-aa7c-f33c8ca3461e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:35.830781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc19653-eb73-44ed-aa7c-f33c8ca3461e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:35.849351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 404/818] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:35.850343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc19653-eb73-44ed-aa7c-f33c8ca3461e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:35.868683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:35.942342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e440d25-fd79-4e24-b2dd-67ef760c0e62 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:35.968792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:35.989035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc19653-eb73-44ed-aa7c-f33c8ca3461e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:35.989276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d243e-a19f-43c3-9b72-1c1e73d154ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:35.989802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d243e-a19f-43c3-9b72-1c1e73d154ef None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:35.990082 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d243e-a19f-43c3-9b72-1c1e73d154ef None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:35.990354 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d243e-a19f-43c3-9b72-1c1e73d154ef None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:35.991065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d243e-a19f-43c3-9b72-1c1e73d154ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:35.991369 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d243e-a19f-43c3-9b72-1c1e73d154ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:35.992736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d243e-a19f-43c3-9b72-1c1e73d154ef None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:35.999101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 415/819] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jul 14 11:24:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:35.999940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d243e-a19f-43c3-9b72-1c1e73d154ef None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:36.030713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284db14e-c6da-4624-9362-d43f7102aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:36.031513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284db14e-c6da-4624-9362-d43f7102aeff None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:36.031793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284db14e-c6da-4624-9362-d43f7102aeff None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:36.032065 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284db14e-c6da-4624-9362-d43f7102aeff None admin] PATH_INFO: `/v3/services/network` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:36.032727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284db14e-c6da-4624-9362-d43f7102aeff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:36.034142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284db14e-c6da-4624-9362-d43f7102aeff None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-284db14e-c6da-4624-9362-d43f7102aeff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:36.048977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:36.050178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 11:24:36.051110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:36.054255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 405/820] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:35 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:36.056370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284db14e-c6da-4624-9362-d43f7102aeff None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:36.128318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d243e-a19f-43c3-9b72-1c1e73d154ef None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:36.141868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:36.172592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd9750-6db0-4663-bbdd-af6ccc869043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:36.174184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd9750-6db0-4663-bbdd-af6ccc869043 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:36.174706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd9750-6db0-4663-bbdd-af6ccc869043 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:36.175692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd9750-6db0-4663-bbdd-af6ccc869043 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:36.188232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd9750-6db0-4663-bbdd-af6ccc869043 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:36.189434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284db14e-c6da-4624-9362-d43f7102aeff None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:36.190423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd9750-6db0-4663-bbdd-af6ccc869043 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-affd9750-6db0-4663-bbdd-af6ccc869043 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:36.211799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:36.212783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:36.213783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:36.213783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:36.213783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:36.213783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:36.213783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:36.213783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:36.213783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:36.213783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:36.213783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:24:36.213783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:36.216944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 416/821] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jul 14 11:24:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:36.218006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd9750-6db0-4663-bbdd-af6ccc869043 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:36.239143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a28c93-bfbd-4b0e-9989-777e1e7539ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:36.240493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a28c93-bfbd-4b0e-9989-777e1e7539ee None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:36.240920 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a28c93-bfbd-4b0e-9989-777e1e7539ee None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:36.241409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a28c93-bfbd-4b0e-9989-777e1e7539ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:36.242480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a28c93-bfbd-4b0e-9989-777e1e7539ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:36.244680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a28c93-bfbd-4b0e-9989-777e1e7539ee None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:36.343746 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd9750-6db0-4663-bbdd-af6ccc869043 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:36.386309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c09f9-f9f1-401e-9335-112ce280bcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:36.386983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c09f9-f9f1-401e-9335-112ce280bcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:36.387390 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c09f9-f9f1-401e-9335-112ce280bcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:36.387657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c09f9-f9f1-401e-9335-112ce280bcd4 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:36.388486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c09f9-f9f1-401e-9335-112ce280bcd4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:36.388790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c09f9-f9f1-401e-9335-112ce280bcd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:36.390230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c09f9-f9f1-401e-9335-112ce280bcd4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:36.405326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 417/822] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:36.405326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c09f9-f9f1-401e-9335-112ce280bcd4 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:36.406415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 406/823] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jul 14 11:24:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:36.407636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a28c93-bfbd-4b0e-9989-777e1e7539ee None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:36.415716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:36.484973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:36.640146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c09f9-f9f1-401e-9335-112ce280bcd4 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:36.642128 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a28c93-bfbd-4b0e-9989-777e1e7539ee None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:36.679953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac4234-78c7-4440-8507-2797891a4df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:36.680725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac4234-78c7-4440-8507-2797891a4df4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:36.681011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac4234-78c7-4440-8507-2797891a4df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:36.681315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac4234-78c7-4440-8507-2797891a4df4 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:36.682024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac4234-78c7-4440-8507-2797891a4df4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:36.682328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac4234-78c7-4440-8507-2797891a4df4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:36.682937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbdefe-f3cb-4105-b010-a9495cafc0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:36.683687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac4234-78c7-4440-8507-2797891a4df4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:36.684055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbdefe-f3cb-4105-b010-a9495cafc0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:36.684539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbdefe-f3cb-4105-b010-a9495cafc0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:36.685013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbdefe-f3cb-4105-b010-a9495cafc0c2 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:36.686076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbdefe-f3cb-4105-b010-a9495cafc0c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:36.686604 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbdefe-f3cb-4105-b010-a9495cafc0c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:36.688216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbdefe-f3cb-4105-b010-a9495cafc0c2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:36.690301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 407/824] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:36.697250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac4234-78c7-4440-8507-2797891a4df4 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:36.711991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 418/825] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jul 14 11:24:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:36.718914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbdefe-f3cb-4105-b010-a9495cafc0c2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:36.767380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:36.811808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac4234-78c7-4440-8507-2797891a4df4 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:36.818882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:36.830232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbdefe-f3cb-4105-b010-a9495cafc0c2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:36.850556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f698ba-618b-45cf-9b05-95b8892d8aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:36.851943 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f698ba-618b-45cf-9b05-95b8892d8aa6 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:36.852325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f698ba-618b-45cf-9b05-95b8892d8aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:36.852699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f698ba-618b-45cf-9b05-95b8892d8aa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:36.853659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f698ba-618b-45cf-9b05-95b8892d8aa6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:36.855724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f698ba-618b-45cf-9b05-95b8892d8aa6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:36.873079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3cd3f-5c33-4c74-8d4e-59a38094d9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:36.873834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3cd3f-5c33-4c74-8d4e-59a38094d9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:36.874124 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3cd3f-5c33-4c74-8d4e-59a38094d9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:36.874399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3cd3f-5c33-4c74-8d4e-59a38094d9e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:36.957505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 408/826] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jul 14 11:24:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:24:36.964339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3cd3f-5c33-4c74-8d4e-59a38094d9e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:36.966275 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3cd3f-5c33-4c74-8d4e-59a38094d9e3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:37.071580 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:37.084073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 419/827] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 11:24:37.093256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f698ba-618b-45cf-9b05-95b8892d8aa6 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:37.211775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f698ba-618b-45cf-9b05-95b8892d8aa6 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:37.231302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:37.251265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893be7d5-17f2-4cff-b659-d813f5de2d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:37.252472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893be7d5-17f2-4cff-b659-d813f5de2d40 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:37.252991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893be7d5-17f2-4cff-b659-d813f5de2d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:37.254130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893be7d5-17f2-4cff-b659-d813f5de2d40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:37.266291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893be7d5-17f2-4cff-b659-d813f5de2d40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:37.268633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893be7d5-17f2-4cff-b659-d813f5de2d40 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-893be7d5-17f2-4cff-b659-d813f5de2d40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:37.288262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:37.289280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:37.290169 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:37.290169 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:37.290169 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:37.290169 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:37.292554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 409/828] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:37.298015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3cd3f-5c33-4c74-8d4e-59a38094d9e3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:37.411200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3cd3f-5c33-4c74-8d4e-59a38094d9e3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:37.451288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241f60-2a90-4eaf-b3a2-5b974797c1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:37.452347 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241f60-2a90-4eaf-b3a2-5b974797c1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:37.452828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241f60-2a90-4eaf-b3a2-5b974797c1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:37.453403 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241f60-2a90-4eaf-b3a2-5b974797c1f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:37.454346 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241f60-2a90-4eaf-b3a2-5b974797c1f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:37.454860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241f60-2a90-4eaf-b3a2-5b974797c1f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:37.456452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241f60-2a90-4eaf-b3a2-5b974797c1f6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:37.469551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 420/829] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:37.478282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893be7d5-17f2-4cff-b659-d813f5de2d40 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:37.503759 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccac16-8ff1-4714-8be8-5df7a57b009a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:37.504494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccac16-8ff1-4714-8be8-5df7a57b009a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:37.504992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccac16-8ff1-4714-8be8-5df7a57b009a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:37.506883 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 421/830] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:24:37.512418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27227b-8893-493f-8254-ae446d8ad9f4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:37.516031 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27227b-8893-493f-8254-ae446d8ad9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:37.516553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27227b-8893-493f-8254-ae446d8ad9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:37.537076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-dd27227b-8893-493f-8254-ae446d8ad9f4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:37.538700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dd27227b-8893-493f-8254-ae446d8ad9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:37.592416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:37.605420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893be7d5-17f2-4cff-b659-d813f5de2d40 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:37.621247 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 422/831] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:24:37.640224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27227b-8893-493f-8254-ae446d8ad9f4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:37.641265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906f5c0-1478-4c3d-99f1-48287edea09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:37.642071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906f5c0-1478-4c3d-99f1-48287edea09b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:37.642353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906f5c0-1478-4c3d-99f1-48287edea09b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:37.642631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906f5c0-1478-4c3d-99f1-48287edea09b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:37.650831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906f5c0-1478-4c3d-99f1-48287edea09b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:37.652091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906f5c0-1478-4c3d-99f1-48287edea09b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-c906f5c0-1478-4c3d-99f1-48287edea09b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:37.667520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:37.668491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:37.670048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:37.670048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:37.670048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:37.670048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:37.670048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:37.670048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:37.670048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:37.670048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:37.670048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 410/832] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:37.674497 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906f5c0-1478-4c3d-99f1-48287edea09b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:37.737643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:37.758651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27227b-8893-493f-8254-ae446d8ad9f4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:37.807565 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2035ff1-429e-4f00-a5ab-d001362baf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:37.808618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2035ff1-429e-4f00-a5ab-d001362baf48 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:37.809097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2035ff1-429e-4f00-a5ab-d001362baf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:37.809617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2035ff1-429e-4f00-a5ab-d001362baf48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:37.813515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906f5c0-1478-4c3d-99f1-48287edea09b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:37.821780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2035ff1-429e-4f00-a5ab-d001362baf48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f2035ff1-429e-4f00-a5ab-d001362baf48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:37.830440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:37.831951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:24:37.832793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:37.840157 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 423/833] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:24:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:37.841678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2035ff1-429e-4f00-a5ab-d001362baf48 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:37.853427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7da52-7d1d-4693-a42b-bc0d88dd4fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:37.854171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7da52-7d1d-4693-a42b-bc0d88dd4fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:37.854465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7da52-7d1d-4693-a42b-bc0d88dd4fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:37.854740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7da52-7d1d-4693-a42b-bc0d88dd4fe9 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:37.856016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7da52-7d1d-4693-a42b-bc0d88dd4fe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:37.856331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7da52-7d1d-4693-a42b-bc0d88dd4fe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:37.858397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7da52-7d1d-4693-a42b-bc0d88dd4fe9 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:37.880859 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 411/834] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:37.881972 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7da52-7d1d-4693-a42b-bc0d88dd4fe9 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:37.896848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:37.960096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2035ff1-429e-4f00-a5ab-d001362baf48 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:37.991432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:37.996446 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61c5da-f3e2-407a-ba0e-d982275a2095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:37.997128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61c5da-f3e2-407a-ba0e-d982275a2095 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:37.997471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61c5da-f3e2-407a-ba0e-d982275a2095 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.000064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61c5da-f3e2-407a-ba0e-d982275a2095 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.000064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61c5da-f3e2-407a-ba0e-d982275a2095 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:38.000064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61c5da-f3e2-407a-ba0e-d982275a2095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.000297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61c5da-f3e2-407a-ba0e-d982275a2095 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.006150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7da52-7d1d-4693-a42b-bc0d88dd4fe9 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.009598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 424/835] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:38.015012 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61c5da-f3e2-407a-ba0e-d982275a2095 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.043612 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb2fe8-7f55-43c9-ae38-f8e058b5e0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.044396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb2fe8-7f55-43c9-ae38-f8e058b5e0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.044683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb2fe8-7f55-43c9-ae38-f8e058b5e0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.044958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb2fe8-7f55-43c9-ae38-f8e058b5e0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.045843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb2fe8-7f55-43c9-ae38-f8e058b5e0c6 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=147d764ef17042dbb7ea179c29711588) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:38.046156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb2fe8-7f55-43c9-ae38-f8e058b5e0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.047598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb2fe8-7f55-43c9-ae38-f8e058b5e0c6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.053841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 412/836] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:37 2022] GET /identity/v3/role_assignments?user.id=147d764ef17042dbb7ea179c29711588&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:38.055398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02075714-d5db-41e4-98e0-ad07983a0ca7 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.055748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02075714-d5db-41e4-98e0-ad07983a0ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.056094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02075714-d5db-41e4-98e0-ad07983a0ca7 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.057738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 413/837] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:38.062699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86995ba-d5ff-46cb-87d2-a2ebf616079a None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.063111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86995ba-d5ff-46cb-87d2-a2ebf616079a None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.063486 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86995ba-d5ff-46cb-87d2-a2ebf616079a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.084083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e86995ba-d5ff-46cb-87d2-a2ebf616079a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:38.085624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e86995ba-d5ff-46cb-87d2-a2ebf616079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:38.143152 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61c5da-f3e2-407a-ba0e-d982275a2095 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.145713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:38.174503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 414/838] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:38.192812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e1da5-7a25-4fdd-a6f6-e9ab2044aaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.193963 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e1da5-7a25-4fdd-a6f6-e9ab2044aaaf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.194394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e1da5-7a25-4fdd-a6f6-e9ab2044aaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.194869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e1da5-7a25-4fdd-a6f6-e9ab2044aaaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.202270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86995ba-d5ff-46cb-87d2-a2ebf616079a None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.207892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e1da5-7a25-4fdd-a6f6-e9ab2044aaaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.209640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e1da5-7a25-4fdd-a6f6-e9ab2044aaaf None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-575e1da5-7a25-4fdd-a6f6-e9ab2044aaaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:38.228414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:38.229589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:38.230709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:38.230709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:38.230709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:38.230709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:38.232608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 425/839] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:38.239442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e1da5-7a25-4fdd-a6f6-e9ab2044aaaf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.310301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:38.330766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86995ba-d5ff-46cb-87d2-a2ebf616079a None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.368285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e1da5-7a25-4fdd-a6f6-e9ab2044aaaf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.369360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75ec8e-27d8-47a0-80fd-bdf12b92edc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.371003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75ec8e-27d8-47a0-80fd-bdf12b92edc3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.371495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75ec8e-27d8-47a0-80fd-bdf12b92edc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.371994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75ec8e-27d8-47a0-80fd-bdf12b92edc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.385405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75ec8e-27d8-47a0-80fd-bdf12b92edc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-0e75ec8e-27d8-47a0-80fd-bdf12b92edc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:38.393784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:38.394836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:38.395721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 415/840] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:38.401358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75ec8e-27d8-47a0-80fd-bdf12b92edc3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.414649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2411ab-4f58-40fd-9645-a0b68cdb5b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.416131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2411ab-4f58-40fd-9645-a0b68cdb5b22 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.416926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2411ab-4f58-40fd-9645-a0b68cdb5b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.417508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2411ab-4f58-40fd-9645-a0b68cdb5b22 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.418813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2411ab-4f58-40fd-9645-a0b68cdb5b22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:38.419886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2411ab-4f58-40fd-9645-a0b68cdb5b22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.422103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2411ab-4f58-40fd-9645-a0b68cdb5b22 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.439218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 426/841] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:38.444903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2411ab-4f58-40fd-9645-a0b68cdb5b22 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.473256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:38.519691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75ec8e-27d8-47a0-80fd-bdf12b92edc3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.557376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292fb20-94c2-44bb-b18d-811bad507649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.558032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292fb20-94c2-44bb-b18d-811bad507649 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.558319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292fb20-94c2-44bb-b18d-811bad507649 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.558598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292fb20-94c2-44bb-b18d-811bad507649 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.559471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292fb20-94c2-44bb-b18d-811bad507649 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:38.559784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292fb20-94c2-44bb-b18d-811bad507649 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.561326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292fb20-94c2-44bb-b18d-811bad507649 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.566382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2411ab-4f58-40fd-9645-a0b68cdb5b22 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.575510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 416/842] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:38.576466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292fb20-94c2-44bb-b18d-811bad507649 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.580423 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:38.603688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f1a4e-d2cf-4299-907d-caaa6f38a97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.604746 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f1a4e-d2cf-4299-907d-caaa6f38a97e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.605193 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f1a4e-d2cf-4299-907d-caaa6f38a97e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.605629 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f1a4e-d2cf-4299-907d-caaa6f38a97e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.614431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f1a4e-d2cf-4299-907d-caaa6f38a97e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.616076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f1a4e-d2cf-4299-907d-caaa6f38a97e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4f9f1a4e-d2cf-4299-907d-caaa6f38a97e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:38.633339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:38.634510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:38.635645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:38.635645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:38.635645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:38.635645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:38.635645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:38.635645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:38.635645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:38.635645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:38.635645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:38.635645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:38.642290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 427/843] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:38.643854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f1a4e-d2cf-4299-907d-caaa6f38a97e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.698495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292fb20-94c2-44bb-b18d-811bad507649 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.739442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:38.745926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9b4c8-a5f7-44ae-ba5e-5c2c52df8ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.746736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9b4c8-a5f7-44ae-ba5e-5c2c52df8ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.747020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9b4c8-a5f7-44ae-ba5e-5c2c52df8ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.747295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9b4c8-a5f7-44ae-ba5e-5c2c52df8ef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.762563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9b4c8-a5f7-44ae-ba5e-5c2c52df8ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.764220 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9b4c8-a5f7-44ae-ba5e-5c2c52df8ef7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.771807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f1a4e-d2cf-4299-907d-caaa6f38a97e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-15e9b4c8-a5f7-44ae-ba5e-5c2c52df8ef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:38.787472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:38.788412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:38.789358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:38.789358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:38.789358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:38.789358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:38.793553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 417/844] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:38.794470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9b4c8-a5f7-44ae-ba5e-5c2c52df8ef7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.824385 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f5a1d-e8c4-4d6a-b880-accef91c4539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.826113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f5a1d-e8c4-4d6a-b880-accef91c4539 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.826604 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f5a1d-e8c4-4d6a-b880-accef91c4539 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.827123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f5a1d-e8c4-4d6a-b880-accef91c4539 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.828287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f5a1d-e8c4-4d6a-b880-accef91c4539 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:38.828919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f5a1d-e8c4-4d6a-b880-accef91c4539 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.830657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f5a1d-e8c4-4d6a-b880-accef91c4539 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.851406 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 428/845] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:38.857457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f5a1d-e8c4-4d6a-b880-accef91c4539 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.871792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:38.919044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9b4c8-a5f7-44ae-ba5e-5c2c52df8ef7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.960272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b04b0-8666-4fb5-a587-cd4237d44999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.961092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b04b0-8666-4fb5-a587-cd4237d44999 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.961409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b04b0-8666-4fb5-a587-cd4237d44999 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.961689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b04b0-8666-4fb5-a587-cd4237d44999 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.962422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b04b0-8666-4fb5-a587-cd4237d44999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:38.962722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b04b0-8666-4fb5-a587-cd4237d44999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.964104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b04b0-8666-4fb5-a587-cd4237d44999 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.971939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f5a1d-e8c4-4d6a-b880-accef91c4539 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.980881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 418/846] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:38.980881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72ae1d-f8e1-444a-a005-b5e48329f5ee None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.980881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72ae1d-f8e1-444a-a005-b5e48329f5ee None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.980881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72ae1d-f8e1-444a-a005-b5e48329f5ee None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.980881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 419/847] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:38.982109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72ae1d-f8e1-444a-a005-b5e48329f5ee None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.015414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:39.021877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdac5a-66c0-4c9d-8ddd-e275faa78446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.022471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdac5a-66c0-4c9d-8ddd-e275faa78446 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.022753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdac5a-66c0-4c9d-8ddd-e275faa78446 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.023024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdac5a-66c0-4c9d-8ddd-e275faa78446 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.090206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72ae1d-f8e1-444a-a005-b5e48329f5ee None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.095254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdac5a-66c0-4c9d-8ddd-e275faa78446 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.096702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdac5a-66c0-4c9d-8ddd-e275faa78446 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.107955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:39.126964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c1440-271c-42d4-bc53-53f61ab17a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.127760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c1440-271c-42d4-bc53-53f61ab17a0c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.128039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c1440-271c-42d4-bc53-53f61ab17a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.128858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c1440-271c-42d4-bc53-53f61ab17a0c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.137515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c1440-271c-42d4-bc53-53f61ab17a0c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.138970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c1440-271c-42d4-bc53-53f61ab17a0c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-f57c1440-271c-42d4-bc53-53f61ab17a0c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:39.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:39.155190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:39.156164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 420/848] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:39.161401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88518fe8-54cd-4e04-bc57-cb3fa44894db None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.162102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88518fe8-54cd-4e04-bc57-cb3fa44894db None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.162706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88518fe8-54cd-4e04-bc57-cb3fa44894db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.180466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-88518fe8-54cd-4e04-bc57-cb3fa44894db None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:39.181719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-88518fe8-54cd-4e04-bc57-cb3fa44894db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:39.207076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 429/849] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:24:39.208127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdac5a-66c0-4c9d-8ddd-e275faa78446 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.245918 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:39.265219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 421/850] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:39.265219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88518fe8-54cd-4e04-bc57-cb3fa44894db None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.326614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdac5a-66c0-4c9d-8ddd-e275faa78446 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.365931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eff862-4893-4669-a03c-f86e8adac1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.366699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eff862-4893-4669-a03c-f86e8adac1ec None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.366877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eff862-4893-4669-a03c-f86e8adac1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.367151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eff862-4893-4669-a03c-f86e8adac1ec None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.367976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eff862-4893-4669-a03c-f86e8adac1ec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:39.368279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eff862-4893-4669-a03c-f86e8adac1ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.369856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eff862-4893-4669-a03c-f86e8adac1ec None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.388360 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 430/851] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jul 14 11:24:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:39.389483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88518fe8-54cd-4e04-bc57-cb3fa44894db None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.393523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eff862-4893-4669-a03c-f86e8adac1ec None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.398125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:39.426830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ecc04-593f-41cf-8383-40260bc41e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.427780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ecc04-593f-41cf-8383-40260bc41e2b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.428156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ecc04-593f-41cf-8383-40260bc41e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.428531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ecc04-593f-41cf-8383-40260bc41e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.440410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ecc04-593f-41cf-8383-40260bc41e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.442076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ecc04-593f-41cf-8383-40260bc41e2b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-936ecc04-593f-41cf-8383-40260bc41e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:39.461674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:39.462835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:39.466186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:39.466186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:39.466186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:39.466186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:39.466186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 422/852] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:39.466186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ecc04-593f-41cf-8383-40260bc41e2b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.495117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:39.504576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eff862-4893-4669-a03c-f86e8adac1ec None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.540947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7843f45-aeeb-40fc-88bc-f769f93fb5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.541786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7843f45-aeeb-40fc-88bc-f769f93fb5ab None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.542067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7843f45-aeeb-40fc-88bc-f769f93fb5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.542343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7843f45-aeeb-40fc-88bc-f769f93fb5ab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.543003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7843f45-aeeb-40fc-88bc-f769f93fb5ab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.544362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7843f45-aeeb-40fc-88bc-f769f93fb5ab None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-e7843f45-aeeb-40fc-88bc-f769f93fb5ab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:39.556908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:39.558268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 11:24:39.559392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:39.563169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 431/853] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:24:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:39.564116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7843f45-aeeb-40fc-88bc-f769f93fb5ab None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.577974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ecc04-593f-41cf-8383-40260bc41e2b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.614238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e8035-6c53-414e-a6d1-6332ca19b54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.615213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e8035-6c53-414e-a6d1-6332ca19b54a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.615734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e8035-6c53-414e-a6d1-6332ca19b54a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.616100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e8035-6c53-414e-a6d1-6332ca19b54a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.645862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:39.670699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7843f45-aeeb-40fc-88bc-f769f93fb5ab None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.691706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e8035-6c53-414e-a6d1-6332ca19b54a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.693484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e8035-6c53-414e-a6d1-6332ca19b54a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.711367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05ddf-3bb4-4836-abde-8807903192ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.712329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05ddf-3bb4-4836-abde-8807903192ce None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.712615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05ddf-3bb4-4836-abde-8807903192ce None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.712893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05ddf-3bb4-4836-abde-8807903192ce None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.713656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e05ddf-3bb4-4836-abde-8807903192ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:39.713965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e05ddf-3bb4-4836-abde-8807903192ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.715777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e05ddf-3bb4-4836-abde-8807903192ce None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.728182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 432/854] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:39.729075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05ddf-3bb4-4836-abde-8807903192ce None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.747966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:39.807974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 423/855] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:24:39.808919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e8035-6c53-414e-a6d1-6332ca19b54a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.850707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05ddf-3bb4-4836-abde-8807903192ce None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.875974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:39.895824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df681c-ccc5-4e9b-bc4f-a7c045390955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.896495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df681c-ccc5-4e9b-bc4f-a7c045390955 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.896783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df681c-ccc5-4e9b-bc4f-a7c045390955 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.897056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df681c-ccc5-4e9b-bc4f-a7c045390955 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.897807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df681c-ccc5-4e9b-bc4f-a7c045390955 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:39.898113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df681c-ccc5-4e9b-bc4f-a7c045390955 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.899478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df681c-ccc5-4e9b-bc4f-a7c045390955 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.904701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 433/856] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:24:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:39.905564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df681c-ccc5-4e9b-bc4f-a7c045390955 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.919342 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e8035-6c53-414e-a6d1-6332ca19b54a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.957801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96783f28-11ab-4f0e-934b-a5d7ad1ab96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.958534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96783f28-11ab-4f0e-934b-a5d7ad1ab96c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.958874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96783f28-11ab-4f0e-934b-a5d7ad1ab96c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.959153 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96783f28-11ab-4f0e-934b-a5d7ad1ab96c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.966204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96783f28-11ab-4f0e-934b-a5d7ad1ab96c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.967681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96783f28-11ab-4f0e-934b-a5d7ad1ab96c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-96783f28-11ab-4f0e-934b-a5d7ad1ab96c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:39.982591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:39.983539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:39.984452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:39.984452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:39.984452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:39.984452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:39.984452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:39.984452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:39.984452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:39.984452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:39.984452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 424/857] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:39.984966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96783f28-11ab-4f0e-934b-a5d7ad1ab96c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.000946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:40.024115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df681c-ccc5-4e9b-bc4f-a7c045390955 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.063262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005cfee-abd3-4deb-8979-6fbd6b0208c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.064118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005cfee-abd3-4deb-8979-6fbd6b0208c2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.064403 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005cfee-abd3-4deb-8979-6fbd6b0208c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.064691 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005cfee-abd3-4deb-8979-6fbd6b0208c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.076283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005cfee-abd3-4deb-8979-6fbd6b0208c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.077777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005cfee-abd3-4deb-8979-6fbd6b0208c2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.091404 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96783f28-11ab-4f0e-934b-a5d7ad1ab96c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-7005cfee-abd3-4deb-8979-6fbd6b0208c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:40.099286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:40.100189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:40.101140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:40.101140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:40.101140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:40.101140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:40.101834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 434/858] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:40.102314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005cfee-abd3-4deb-8979-6fbd6b0208c2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.111837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:40.130425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a20e-5a35-427f-84c1-7cbe62396d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.131098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a20e-5a35-427f-84c1-7cbe62396d8b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.131378 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a20e-5a35-427f-84c1-7cbe62396d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.131650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a20e-5a35-427f-84c1-7cbe62396d8b None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.132362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5a20e-5a35-427f-84c1-7cbe62396d8b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:40.132670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5a20e-5a35-427f-84c1-7cbe62396d8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.134074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5a20e-5a35-427f-84c1-7cbe62396d8b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.138768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 425/859] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:24:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:40.147703 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a20e-5a35-427f-84c1-7cbe62396d8b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.245489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005cfee-abd3-4deb-8979-6fbd6b0208c2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.271543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a20e-5a35-427f-84c1-7cbe62396d8b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.280602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:40.287316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51078a-2ea3-4c83-b795-aaf47257bd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.294505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51078a-2ea3-4c83-b795-aaf47257bd31 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.294505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51078a-2ea3-4c83-b795-aaf47257bd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.294505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51078a-2ea3-4c83-b795-aaf47257bd31 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.294505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51078a-2ea3-4c83-b795-aaf47257bd31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:40.294505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51078a-2ea3-4c83-b795-aaf47257bd31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.294505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51078a-2ea3-4c83-b795-aaf47257bd31 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.311272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 435/860] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:40.312494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cf671-3698-4af1-9f74-30590cf632ec None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.312677 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cf671-3698-4af1-9f74-30590cf632ec None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.313007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cf671-3698-4af1-9f74-30590cf632ec None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.314128 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45df2c6-3e0f-4806-84af-48fad933fc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.318519 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45df2c6-3e0f-4806-84af-48fad933fc29 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.318654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 436/861] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:40.318734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45df2c6-3e0f-4806-84af-48fad933fc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.318734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45df2c6-3e0f-4806-84af-48fad933fc29 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.318734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45df2c6-3e0f-4806-84af-48fad933fc29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:40.318734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45df2c6-3e0f-4806-84af-48fad933fc29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.319008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cf671-3698-4af1-9f74-30590cf632ec None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.319079 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45df2c6-3e0f-4806-84af-48fad933fc29 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.333484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 426/862] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:40.335051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4738d-41a2-4527-bb16-a179234efff2 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.335339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4738d-41a2-4527-bb16-a179234efff2 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.335610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4738d-41a2-4527-bb16-a179234efff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.353839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-37f4738d-41a2-4527-bb16-a179234efff2 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:40.354949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-37f4738d-41a2-4527-bb16-a179234efff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:40.395325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:40.430645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cf671-3698-4af1-9f74-30590cf632ec None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.431256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 427/863] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:40.432001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4738d-41a2-4527-bb16-a179234efff2 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.477931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80508f-2acf-46b6-b972-b159179e0c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.478591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80508f-2acf-46b6-b972-b159179e0c01 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.479508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80508f-2acf-46b6-b972-b159179e0c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.479508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80508f-2acf-46b6-b972-b159179e0c01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.479977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80508f-2acf-46b6-b972-b159179e0c01 None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=0bc624eba860448eab54b516bd344a46) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:40.480289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80508f-2acf-46b6-b972-b159179e0c01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.481747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80508f-2acf-46b6-b972-b159179e0c01 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.489086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 437/864] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/role_assignments?user.id=0bc624eba860448eab54b516bd344a46&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:40.490200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80508f-2acf-46b6-b972-b159179e0c01 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.562645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:40.565881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4738d-41a2-4527-bb16-a179234efff2 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.607687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f66fb-2bbc-4403-afdd-47862c4f92d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.608834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f66fb-2bbc-4403-afdd-47862c4f92d6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.609365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f66fb-2bbc-4403-afdd-47862c4f92d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.609818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f66fb-2bbc-4403-afdd-47862c4f92d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.618263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f66fb-2bbc-4403-afdd-47862c4f92d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.620061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f66fb-2bbc-4403-afdd-47862c4f92d6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.627797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80508f-2acf-46b6-b972-b159179e0c01 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-e79f66fb-2bbc-4403-afdd-47862c4f92d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:40.636173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:40.637120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:40.638251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:40.638251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:40.638251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:40.638251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:40.638251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:40.638251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:40.638251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:40.638251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:40.640449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 428/865] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:40.647221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f66fb-2bbc-4403-afdd-47862c4f92d6 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.668878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808c247-ae86-48ea-a9a4-6383c340bf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.670139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808c247-ae86-48ea-a9a4-6383c340bf78 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.670648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808c247-ae86-48ea-a9a4-6383c340bf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.671133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808c247-ae86-48ea-a9a4-6383c340bf78 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.672055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808c247-ae86-48ea-a9a4-6383c340bf78 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.673732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808c247-ae86-48ea-a9a4-6383c340bf78 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-a808c247-ae86-48ea-a9a4-6383c340bf78 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:40.687119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:40.688101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 11:24:40.688957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:40.689791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:40.692234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 438/866] 10.222.0.40 () {56 vars in 1132 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:40.699217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808c247-ae86-48ea-a9a4-6383c340bf78 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.771738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f66fb-2bbc-4403-afdd-47862c4f92d6 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.818315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20773f18-d722-41ba-89ac-e58a9b082b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.819081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20773f18-d722-41ba-89ac-e58a9b082b2b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.819467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20773f18-d722-41ba-89ac-e58a9b082b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.819847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20773f18-d722-41ba-89ac-e58a9b082b2b None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.820724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20773f18-d722-41ba-89ac-e58a9b082b2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:40.821236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20773f18-d722-41ba-89ac-e58a9b082b2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.822841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20773f18-d722-41ba-89ac-e58a9b082b2b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.828028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808c247-ae86-48ea-a9a4-6383c340bf78 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.836891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 429/867] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:40.843199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20773f18-d722-41ba-89ac-e58a9b082b2b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.846841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:40.865333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f16ad0-6b92-465c-b364-a1ec8b621b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.866811 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f16ad0-6b92-465c-b364-a1ec8b621b98 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.867290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f16ad0-6b92-465c-b364-a1ec8b621b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.867755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f16ad0-6b92-465c-b364-a1ec8b621b98 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.868647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f16ad0-6b92-465c-b364-a1ec8b621b98 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:40.869199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f16ad0-6b92-465c-b364-a1ec8b621b98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.871027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f16ad0-6b92-465c-b364-a1ec8b621b98 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.877942 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 439/868] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:40.884617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f16ad0-6b92-465c-b364-a1ec8b621b98 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.944136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:40.953984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20773f18-d722-41ba-89ac-e58a9b082b2b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:41.001767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2841b9-5ec9-429d-a43f-d483f06a7faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:41.003106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2841b9-5ec9-429d-a43f-d483f06a7faa None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:41.003512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2841b9-5ec9-429d-a43f-d483f06a7faa None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:41.003866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2841b9-5ec9-429d-a43f-d483f06a7faa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:41.005125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2841b9-5ec9-429d-a43f-d483f06a7faa None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=42a5ec793f014cefb9d0a0d7258bbb59) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:41.005527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2841b9-5ec9-429d-a43f-d483f06a7faa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:41.009006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2841b9-5ec9-429d-a43f-d483f06a7faa None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:41.016252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f16ad0-6b92-465c-b364-a1ec8b621b98 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:41.017703 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 430/869] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/role_assignments?user.id=42a5ec793f014cefb9d0a0d7258bbb59&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:41.065433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29bd2b-eb1c-4d68-9555-d2043d64c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:41.066437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29bd2b-eb1c-4d68-9555-d2043d64c99b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:41.066929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29bd2b-eb1c-4d68-9555-d2043d64c99b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:41.067382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29bd2b-eb1c-4d68-9555-d2043d64c99b None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:41.068283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29bd2b-eb1c-4d68-9555-d2043d64c99b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:41.068767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29bd2b-eb1c-4d68-9555-d2043d64c99b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:41.071314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29bd2b-eb1c-4d68-9555-d2043d64c99b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:41.078749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 440/870] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:42.282311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69efe-cfa2-43ba-838b-f6cf6211f7b8 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:42.282837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69efe-cfa2-43ba-838b-f6cf6211f7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:42.282837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69efe-cfa2-43ba-838b-f6cf6211f7b8 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:42.284535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 431/871] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:42.290161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c597c-3d65-4fe4-bae1-8b4dcdae6207 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:42.290625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c597c-3d65-4fe4-bae1-8b4dcdae6207 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:42.290723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c597c-3d65-4fe4-bae1-8b4dcdae6207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:42.307701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7b3c597c-3d65-4fe4-bae1-8b4dcdae6207 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:42.308884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7b3c597c-3d65-4fe4-bae1-8b4dcdae6207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:42.353830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:42.383170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 441/872] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:42.404430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca69efe-cfa2-43ba-838b-f6cf6211f7b8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:42.515986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca69efe-cfa2-43ba-838b-f6cf6211f7b8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:42.552961 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265b463-cefd-41d8-9fe8-4d43b2b45492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:42.553999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265b463-cefd-41d8-9fe8-4d43b2b45492 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:42.554489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265b463-cefd-41d8-9fe8-4d43b2b45492 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:42.554984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265b463-cefd-41d8-9fe8-4d43b2b45492 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:42.555926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265b463-cefd-41d8-9fe8-4d43b2b45492 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:42.557576 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265b463-cefd-41d8-9fe8-4d43b2b45492 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:42.659993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 432/873] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:24:42 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:42.786840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02b5e9-c115-4956-a2de-0a3b370f051a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:42.787198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02b5e9-c115-4956-a2de-0a3b370f051a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:42.787393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02b5e9-c115-4956-a2de-0a3b370f051a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:42.789063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 442/874] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:42.794813 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea001fd-180d-45f7-a676-1f0c91480f9b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:42.795307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea001fd-180d-45f7-a676-1f0c91480f9b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:42.795686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea001fd-180d-45f7-a676-1f0c91480f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:42.818351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-cea001fd-180d-45f7-a676-1f0c91480f9b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:42.819903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cea001fd-180d-45f7-a676-1f0c91480f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:42.832723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:42.909508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 433/875] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:42 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:42.933426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02b5e9-c115-4956-a2de-0a3b370f051a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:43.044205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02b5e9-c115-4956-a2de-0a3b370f051a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:43.082384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645c3d6-05a5-4c09-b7b1-007ee3a25bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:43.083180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645c3d6-05a5-4c09-b7b1-007ee3a25bbf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.083468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645c3d6-05a5-4c09-b7b1-007ee3a25bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.083740 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645c3d6-05a5-4c09-b7b1-007ee3a25bbf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.095185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645c3d6-05a5-4c09-b7b1-007ee3a25bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-6645c3d6-05a5-4c09-b7b1-007ee3a25bbf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:43.103989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:43.104953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:43.105839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 443/876] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jul 14 11:24:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:43.112094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886176fd-67f4-462b-a71f-5be113276e7e None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.112094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886176fd-67f4-462b-a71f-5be113276e7e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.112094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886176fd-67f4-462b-a71f-5be113276e7e None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.113184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 434/877] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:43.117190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645c3d6-05a5-4c09-b7b1-007ee3a25bbf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:43.117826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a2c57-dfe9-439f-a6a1-f00399104f7f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.118115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a2c57-dfe9-439f-a6a1-f00399104f7f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.118385 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a2c57-dfe9-439f-a6a1-f00399104f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.136888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-300a2c57-dfe9-439f-a6a1-f00399104f7f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:43.138111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-300a2c57-dfe9-439f-a6a1-f00399104f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:43.153922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:43.221261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 435/878] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:24:43.223402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7462353-c269-4dd3-ac5d-994a5e3bd886 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.223758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7462353-c269-4dd3-ac5d-994a5e3bd886 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.224119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7462353-c269-4dd3-ac5d-994a5e3bd886 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.225861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 436/879] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:43.230853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d017d0-0754-4648-9b1b-ba3e29f38b99 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.231130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d017d0-0754-4648-9b1b-ba3e29f38b99 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.231543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d017d0-0754-4648-9b1b-ba3e29f38b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.232482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645c3d6-05a5-4c09-b7b1-007ee3a25bbf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:43.250545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-33d017d0-0754-4648-9b1b-ba3e29f38b99 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:43.251744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-33d017d0-0754-4648-9b1b-ba3e29f38b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:43.270053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37621440-50e6-4fc4-93c6-99668c326b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:43.270726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37621440-50e6-4fc4-93c6-99668c326b27 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.271002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37621440-50e6-4fc4-93c6-99668c326b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.271276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37621440-50e6-4fc4-93c6-99668c326b27 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.272016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37621440-50e6-4fc4-93c6-99668c326b27 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:43.272318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37621440-50e6-4fc4-93c6-99668c326b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:43.273735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37621440-50e6-4fc4-93c6-99668c326b27 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:43.282114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 444/880] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jul 14 11:24:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:43.283215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37621440-50e6-4fc4-93c6-99668c326b27 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:43.335443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 437/881] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:43.336338 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d017d0-0754-4648-9b1b-ba3e29f38b99 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:43.338853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:43.429742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:43.436784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37621440-50e6-4fc4-93c6-99668c326b27 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:43.482995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d017d0-0754-4648-9b1b-ba3e29f38b99 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:43.491146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0166ba-154a-46bd-a06e-534f2cc5e8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:43.492398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0166ba-154a-46bd-a06e-534f2cc5e8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.492851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0166ba-154a-46bd-a06e-534f2cc5e8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.493323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0166ba-154a-46bd-a06e-534f2cc5e8d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.494426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0166ba-154a-46bd-a06e-534f2cc5e8d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:43.496744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0166ba-154a-46bd-a06e-534f2cc5e8d6 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4f0166ba-154a-46bd-a06e-534f2cc5e8d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:43.518336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:43.519975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:43.520813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 445/882] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:43.522351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0166ba-154a-46bd-a06e-534f2cc5e8d6 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:43.545428 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b4102-cc7b-4bf3-8026-cef18285446d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:43.546480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b4102-cc7b-4bf3-8026-cef18285446d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.546847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b4102-cc7b-4bf3-8026-cef18285446d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.547126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b4102-cc7b-4bf3-8026-cef18285446d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.559244 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b4102-cc7b-4bf3-8026-cef18285446d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:43.560708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b4102-cc7b-4bf3-8026-cef18285446d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-d84b4102-cc7b-4bf3-8026-cef18285446d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:43.580091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:43.582358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:43.585117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:43.585117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:43.585117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:43.585117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:43.585117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 438/883] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:43.585117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b4102-cc7b-4bf3-8026-cef18285446d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:43.647870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0166ba-154a-46bd-a06e-534f2cc5e8d6 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:43.650059 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:43.692968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180c72-1f90-4a00-b2b7-cd5be9819558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:43.693805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180c72-1f90-4a00-b2b7-cd5be9819558 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.693995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180c72-1f90-4a00-b2b7-cd5be9819558 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.694264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180c72-1f90-4a00-b2b7-cd5be9819558 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.694993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c180c72-1f90-4a00-b2b7-cd5be9819558 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:43.696368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c180c72-1f90-4a00-b2b7-cd5be9819558 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:43.708794 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b4102-cc7b-4bf3-8026-cef18285446d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:43.743356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83471a-be28-4727-92e8-f693b49390fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:43.744017 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83471a-be28-4727-92e8-f693b49390fd None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.744299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83471a-be28-4727-92e8-f693b49390fd None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.744572 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83471a-be28-4727-92e8-f693b49390fd None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.745347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83471a-be28-4727-92e8-f693b49390fd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:43.745607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83471a-be28-4727-92e8-f693b49390fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:43.746966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83471a-be28-4727-92e8-f693b49390fd None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:43.751538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 439/884] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jul 14 11:24:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:43.752307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83471a-be28-4727-92e8-f693b49390fd None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:43.813538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 446/885] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:24:43 2022] POST /identity/v3/services => generated 269 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:43.814426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180c72-1f90-4a00-b2b7-cd5be9819558 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:43.854719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:43.896139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83471a-be28-4727-92e8-f693b49390fd None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:43.917571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:43.952131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d0a3b-6517-410b-b970-202da9322a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:43.952978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d0a3b-6517-410b-b970-202da9322a3b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:43.953395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d0a3b-6517-410b-b970-202da9322a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:43.953777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d0a3b-6517-410b-b970-202da9322a3b None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:43.955392 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d0a3b-6517-410b-b970-202da9322a3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:43.956204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d0a3b-6517-410b-b970-202da9322a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:43.958487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d0a3b-6517-410b-b970-202da9322a3b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:43.970607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180c72-1f90-4a00-b2b7-cd5be9819558 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:43.975791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 440/886] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:43.981931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d0a3b-6517-410b-b970-202da9322a3b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:44.031584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dd3f8-62fb-485f-b623-51a71c8699ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:44.032632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dd3f8-62fb-485f-b623-51a71c8699ef None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:44.032994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dd3f8-62fb-485f-b623-51a71c8699ef None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:44.033311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dd3f8-62fb-485f-b623-51a71c8699ef None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:44.034060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dd3f8-62fb-485f-b623-51a71c8699ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:44.034400 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dd3f8-62fb-485f-b623-51a71c8699ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:44.035789 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dd3f8-62fb-485f-b623-51a71c8699ef None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:44.043650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 447/887] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jul 14 11:24:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:44.112150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d0a3b-6517-410b-b970-202da9322a3b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:44.152350 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b75d8-d11f-47d8-870e-e40d9ab88ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:44.153198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b75d8-d11f-47d8-870e-e40d9ab88ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:44.153485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b75d8-d11f-47d8-870e-e40d9ab88ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:44.153761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b75d8-d11f-47d8-870e-e40d9ab88ab5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:44.162344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b75d8-d11f-47d8-870e-e40d9ab88ab5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:44.163835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b75d8-d11f-47d8-870e-e40d9ab88ab5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-a06b75d8-d11f-47d8-870e-e40d9ab88ab5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:44.181151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:44.182193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:44.183150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:44.183150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:44.183150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:44.183150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:44.183150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:44.183150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:44.183150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:44.183150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:44.183150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:44.183150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:44.186793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 441/888] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jul 14 11:24:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:44.188847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dd3f8-62fb-485f-b623-51a71c8699ef None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:44.278852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:44.294238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dd3f8-62fb-485f-b623-51a71c8699ef None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:44.334483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdc4cb-9e67-49bf-90d0-04795e3c56f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:44.335161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdc4cb-9e67-49bf-90d0-04795e3c56f3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:44.335446 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdc4cb-9e67-49bf-90d0-04795e3c56f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:44.335724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdc4cb-9e67-49bf-90d0-04795e3c56f3 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:44.336576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdc4cb-9e67-49bf-90d0-04795e3c56f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:44.336884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdc4cb-9e67-49bf-90d0-04795e3c56f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:44.338350 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdc4cb-9e67-49bf-90d0-04795e3c56f3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:44.359034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 448/889] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jul 14 11:24:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:44.364558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b75d8-d11f-47d8-870e-e40d9ab88ab5 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:44.401897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:44.471227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b75d8-d11f-47d8-870e-e40d9ab88ab5 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:44.507346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcb5c3-7424-4f99-a36a-b9c3c051c275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:44.507997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcb5c3-7424-4f99-a36a-b9c3c051c275 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:44.508281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcb5c3-7424-4f99-a36a-b9c3c051c275 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:44.508555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcb5c3-7424-4f99-a36a-b9c3c051c275 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:44.579249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcb5c3-7424-4f99-a36a-b9c3c051c275 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:44.580714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcb5c3-7424-4f99-a36a-b9c3c051c275 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:44.620357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:44.687370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 442/890] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:44 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:24:44.688862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdc4cb-9e67-49bf-90d0-04795e3c56f3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:44.795889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdc4cb-9e67-49bf-90d0-04795e3c56f3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:44.833101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d778f-981b-49b2-807d-6367ca88fd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:44.833939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d778f-981b-49b2-807d-6367ca88fd99 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:44.834218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d778f-981b-49b2-807d-6367ca88fd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:44.834488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d778f-981b-49b2-807d-6367ca88fd99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:44.845566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d778f-981b-49b2-807d-6367ca88fd99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:44.847058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d778f-981b-49b2-807d-6367ca88fd99 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:44.849912 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-390d778f-981b-49b2-807d-6367ca88fd99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:44.868657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:44.869714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:44.870636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:44.870636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:44.870636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:44.870636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:44.870636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 449/891] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:44.872588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f2ef-67d5-4b74-aa26-db1ed15f4f0d None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:44.872876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f2ef-67d5-4b74-aa26-db1ed15f4f0d None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:44.873172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f2ef-67d5-4b74-aa26-db1ed15f4f0d None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:44.877687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 443/892] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:44.880047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cf357-32d0-4161-9451-826ae5d9cb3f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:44.880408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cf357-32d0-4161-9451-826ae5d9cb3f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:44.881181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cf357-32d0-4161-9451-826ae5d9cb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:44.882879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d778f-981b-49b2-807d-6367ca88fd99 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:44.904462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-538cf357-32d0-4161-9451-826ae5d9cb3f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:44.905748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-538cf357-32d0-4161-9451-826ae5d9cb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:44.992541 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 444/893] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:24:44.992771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d778f-981b-49b2-807d-6367ca88fd99 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:45.015219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cf357-32d0-4161-9451-826ae5d9cb3f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:45.034881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcf99f-a040-474b-b6ee-98d84bc42225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:45.035566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcf99f-a040-474b-b6ee-98d84bc42225 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:45.035849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcf99f-a040-474b-b6ee-98d84bc42225 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:45.036126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcf99f-a040-474b-b6ee-98d84bc42225 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:45.036877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfcf99f-a040-474b-b6ee-98d84bc42225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:45.037213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfcf99f-a040-474b-b6ee-98d84bc42225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:45.039432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfcf99f-a040-474b-b6ee-98d84bc42225 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:45.052627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 450/894] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:45.060012 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcf99f-a040-474b-b6ee-98d84bc42225 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:45.107703 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:45.131619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:45.166584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cf357-32d0-4161-9451-826ae5d9cb3f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:45.184107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcf99f-a040-474b-b6ee-98d84bc42225 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:45.217019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7961e-1623-40c9-b95c-3059002da908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:45.217882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7961e-1623-40c9-b95c-3059002da908 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:45.218169 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7961e-1623-40c9-b95c-3059002da908 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:45.218444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7961e-1623-40c9-b95c-3059002da908 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:45.219124 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7961e-1623-40c9-b95c-3059002da908 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:45.220548 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7961e-1623-40c9-b95c-3059002da908 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:45.233049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f58f65-ab3d-45cc-8ed9-59604566dd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:45.234235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f58f65-ab3d-45cc-8ed9-59604566dd35 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:45.234718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f58f65-ab3d-45cc-8ed9-59604566dd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:45.235190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f58f65-ab3d-45cc-8ed9-59604566dd35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-b9a7961e-1623-40c9-b95c-3059002da908 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:45.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:45.238234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:45.239406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 445/895] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:24:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:45.244147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7961e-1623-40c9-b95c-3059002da908 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:45.247598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f58f65-ab3d-45cc-8ed9-59604566dd35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:45.250270 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f58f65-ab3d-45cc-8ed9-59604566dd35 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-96f58f65-ab3d-45cc-8ed9-59604566dd35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:45.269126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:45.270295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:45.271484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:45.271484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:45.271484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:45.271484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:45.271484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:45.271484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:45.271484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:45.271484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:45.278889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 451/896] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:45.280678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f58f65-ab3d-45cc-8ed9-59604566dd35 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:45.362151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7961e-1623-40c9-b95c-3059002da908 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:45.399540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:45.408832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f58f65-ab3d-45cc-8ed9-59604566dd35 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:45.411523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d317fc-aa6e-4f9a-bd90-554ecffd9244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:45.413447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d317fc-aa6e-4f9a-bd90-554ecffd9244 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:45.413901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d317fc-aa6e-4f9a-bd90-554ecffd9244 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:45.422983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d317fc-aa6e-4f9a-bd90-554ecffd9244 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:45.422983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d317fc-aa6e-4f9a-bd90-554ecffd9244 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:45.422983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d317fc-aa6e-4f9a-bd90-554ecffd9244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:45.422983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d317fc-aa6e-4f9a-bd90-554ecffd9244 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:45.423787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 446/897] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:24:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:45.431417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d317fc-aa6e-4f9a-bd90-554ecffd9244 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:45.437196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:45.461067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1aa1e-a2f0-4626-a1f8-428ec22281f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:45.462862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1aa1e-a2f0-4626-a1f8-428ec22281f9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:45.463257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1aa1e-a2f0-4626-a1f8-428ec22281f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:45.463705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1aa1e-a2f0-4626-a1f8-428ec22281f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:45.464690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1aa1e-a2f0-4626-a1f8-428ec22281f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:45.465721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1aa1e-a2f0-4626-a1f8-428ec22281f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:45.467602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1aa1e-a2f0-4626-a1f8-428ec22281f9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:45.484869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 452/898] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:45.495011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1aa1e-a2f0-4626-a1f8-428ec22281f9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:45.548801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d317fc-aa6e-4f9a-bd90-554ecffd9244 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:45.585555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1d5db-8142-422b-bb76-58aa349e94fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:45.586221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1d5db-8142-422b-bb76-58aa349e94fa None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:45.586500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1d5db-8142-422b-bb76-58aa349e94fa None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:45.586768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1d5db-8142-422b-bb76-58aa349e94fa None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:45.587481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1d5db-8142-422b-bb76-58aa349e94fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:45.587781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1d5db-8142-422b-bb76-58aa349e94fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:45.589295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1d5db-8142-422b-bb76-58aa349e94fa None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:45.594973 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 447/899] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:24:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:45.601081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1d5db-8142-422b-bb76-58aa349e94fa None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:45.602636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1aa1e-a2f0-4626-a1f8-428ec22281f9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:45.637202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fb3c4-6471-408d-8143-48c0849e0bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:45.638090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fb3c4-6471-408d-8143-48c0849e0bbb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:45.638203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fb3c4-6471-408d-8143-48c0849e0bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:45.638412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fb3c4-6471-408d-8143-48c0849e0bbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:45.639252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fb3c4-6471-408d-8143-48c0849e0bbb None admin] RBAC: Adding query filter params (role.id=b7e771a1c5f54085a2a9174024f86f19, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=0bc624eba860448eab54b516bd344a46) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:45.639552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fb3c4-6471-408d-8143-48c0849e0bbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:45.640977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fb3c4-6471-408d-8143-48c0849e0bbb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:45.648135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 453/900] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:45 2022] GET /identity/v3/role_assignments?user.id=0bc624eba860448eab54b516bd344a46&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=b7e771a1c5f54085a2a9174024f86f19 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:45.674195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:45.727509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1d5db-8142-422b-bb76-58aa349e94fa None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:45.764359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40286318-92e4-40e7-bb0d-eec9cfebc134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:45.765040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40286318-92e4-40e7-bb0d-eec9cfebc134 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:45.765350 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40286318-92e4-40e7-bb0d-eec9cfebc134 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:45.765625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40286318-92e4-40e7-bb0d-eec9cfebc134 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:45.766443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40286318-92e4-40e7-bb0d-eec9cfebc134 None admin] RBAC: Adding query filter params (interface=public, service_id=88460b3ae43c46d88a5f84f5e93d95c2, region_id=RegionOne) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:45.766751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40286318-92e4-40e7-bb0d-eec9cfebc134 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:45.768157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40286318-92e4-40e7-bb0d-eec9cfebc134 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:45.775393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 448/901] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jul 14 11:24:45 2022] GET /identity/v3/endpoints?service_id=88460b3ae43c46d88a5f84f5e93d95c2&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:45.780913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fb3c4-6471-408d-8143-48c0849e0bbb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:45.826326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:45.885537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fb3c4-6471-408d-8143-48c0849e0bbb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:45.921179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7deee-ee40-43cc-8075-0eeb1a1708ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:45.921779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7deee-ee40-43cc-8075-0eeb1a1708ef None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:45.922080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7deee-ee40-43cc-8075-0eeb1a1708ef None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:45.922394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7deee-ee40-43cc-8075-0eeb1a1708ef None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:45.923079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7deee-ee40-43cc-8075-0eeb1a1708ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:45.924491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7deee-ee40-43cc-8075-0eeb1a1708ef None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:45.930943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 454/902] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jul 14 11:24:45 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:24:46.036718 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49472c-1a58-43b7-839c-ac5010de6e85 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:46.037076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49472c-1a58-43b7-839c-ac5010de6e85 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:46.037296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49472c-1a58-43b7-839c-ac5010de6e85 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:46.038935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 449/903] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:46.044522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ad710-2113-4991-ac1c-9a89732da666 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:46.044800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ad710-2113-4991-ac1c-9a89732da666 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:46.045073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ad710-2113-4991-ac1c-9a89732da666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:46.063199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6d6ad710-2113-4991-ac1c-9a89732da666 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:46.064402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6d6ad710-2113-4991-ac1c-9a89732da666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:46.137266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 455/904] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:46.160375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49472c-1a58-43b7-839c-ac5010de6e85 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:46.172578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5395bd-0302-4dcc-a753-9a1a96fa20c8 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:46.172879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5395bd-0302-4dcc-a753-9a1a96fa20c8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:46.175237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5395bd-0302-4dcc-a753-9a1a96fa20c8 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:46.176762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 456/905] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:46.182755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0276-133d-456e-aba6-72e786bd30ce None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:46.183297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0276-133d-456e-aba6-72e786bd30ce None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:46.183790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0276-133d-456e-aba6-72e786bd30ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:46.202808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-61fa0276-133d-456e-aba6-72e786bd30ce None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:46.204330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-61fa0276-133d-456e-aba6-72e786bd30ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:46.233328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:46.285078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49472c-1a58-43b7-839c-ac5010de6e85 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:46.285640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 457/906] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:46.302242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa0276-133d-456e-aba6-72e786bd30ce None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:46.306896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:46.324071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c6bf7-10c4-48e7-b1f6-c24dd9b07c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:46.325149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c6bf7-10c4-48e7-b1f6-c24dd9b07c97 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:46.325455 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c6bf7-10c4-48e7-b1f6-c24dd9b07c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:46.325725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c6bf7-10c4-48e7-b1f6-c24dd9b07c97 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:46.326385 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c6bf7-10c4-48e7-b1f6-c24dd9b07c97 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:46.328379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c6bf7-10c4-48e7-b1f6-c24dd9b07c97 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-c57c6bf7-10c4-48e7-b1f6-c24dd9b07c97 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:46.345747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:46.346814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:46.347660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 450/907] 10.222.0.40 () {56 vars in 1132 bytes} [Thu Jul 14 11:24:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:46.353902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c6bf7-10c4-48e7-b1f6-c24dd9b07c97 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:46.420461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa0276-133d-456e-aba6-72e786bd30ce None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:46.465004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f09f8-0cab-43e6-9e29-16d7ed4a37ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:46.465776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f09f8-0cab-43e6-9e29-16d7ed4a37ec None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:46.466215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f09f8-0cab-43e6-9e29-16d7ed4a37ec None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:46.466490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f09f8-0cab-43e6-9e29-16d7ed4a37ec None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:46.467597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f09f8-0cab-43e6-9e29-16d7ed4a37ec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:46.469051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f09f8-0cab-43e6-9e29-16d7ed4a37ec None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:46.486014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c6bf7-10c4-48e7-b1f6-c24dd9b07c97 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:46.525558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e679cd5-8b02-45f8-b697-7560a8e05c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:46.526225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e679cd5-8b02-45f8-b697-7560a8e05c8e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:46.526509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e679cd5-8b02-45f8-b697-7560a8e05c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:46.526876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e679cd5-8b02-45f8-b697-7560a8e05c8e None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:46.527489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e679cd5-8b02-45f8-b697-7560a8e05c8e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:46.527803 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e679cd5-8b02-45f8-b697-7560a8e05c8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:46.529249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e679cd5-8b02-45f8-b697-7560a8e05c8e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:46.538822 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 451/908] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:46.546216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e679cd5-8b02-45f8-b697-7560a8e05c8e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:46.574154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 458/909] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:24:46 2022] POST /identity/v3/services => generated 241 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:24:46.600277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:46.660749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e679cd5-8b02-45f8-b697-7560a8e05c8e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:46.706180 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566596c-a22d-4c82-babe-db6ce94d1f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:46.706863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566596c-a22d-4c82-babe-db6ce94d1f22 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:46.707141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566596c-a22d-4c82-babe-db6ce94d1f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:46.707416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566596c-a22d-4c82-babe-db6ce94d1f22 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:46.708145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566596c-a22d-4c82-babe-db6ce94d1f22 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:46.708457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566596c-a22d-4c82-babe-db6ce94d1f22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:46.709905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566596c-a22d-4c82-babe-db6ce94d1f22 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:46.716311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 452/910] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:46.722757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f09f8-0cab-43e6-9e29-16d7ed4a37ec None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:46.754064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:46.845119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f09f8-0cab-43e6-9e29-16d7ed4a37ec None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:46.882229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac976c-84e2-4999-984c-4f5ee0447e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:46.882894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac976c-84e2-4999-984c-4f5ee0447e5b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:46.883172 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac976c-84e2-4999-984c-4f5ee0447e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:46.883442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac976c-84e2-4999-984c-4f5ee0447e5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:46.884240 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac976c-84e2-4999-984c-4f5ee0447e5b None admin] RBAC: Adding query filter params (interface=public, service_id=9930447fc6e147bfa721c2a62ee73e9b, region_id=RegionOne) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:46.884545 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac976c-84e2-4999-984c-4f5ee0447e5b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:46.885937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac976c-84e2-4999-984c-4f5ee0447e5b None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:46.891949 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 459/911] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jul 14 11:24:46 2022] GET /identity/v3/endpoints?service_id=9930447fc6e147bfa721c2a62ee73e9b&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:46.897211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566596c-a22d-4c82-babe-db6ce94d1f22 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:47.004552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566596c-a22d-4c82-babe-db6ce94d1f22 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:47.013683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:47.038672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03fea0-ab46-4f83-8c83-ad0195b7bb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:47.039334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03fea0-ab46-4f83-8c83-ad0195b7bb5e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:47.039613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03fea0-ab46-4f83-8c83-ad0195b7bb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:47.039887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03fea0-ab46-4f83-8c83-ad0195b7bb5e None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:47.040571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03fea0-ab46-4f83-8c83-ad0195b7bb5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:47.041954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03fea0-ab46-4f83-8c83-ad0195b7bb5e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:47.048380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 453/912] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jul 14 11:24:46 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:24:47.758617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d533c-c3a4-49a4-bb40-d0cbcbc2e6c5 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:47.759028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d533c-c3a4-49a4-bb40-d0cbcbc2e6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:47.759162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d533c-c3a4-49a4-bb40-d0cbcbc2e6c5 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:47.760767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 460/913] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:47.766470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106efe5-b2e8-43eb-b2e7-6c3df076d18d None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:47.766945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106efe5-b2e8-43eb-b2e7-6c3df076d18d None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:47.767023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106efe5-b2e8-43eb-b2e7-6c3df076d18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:47.784520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-5106efe5-b2e8-43eb-b2e7-6c3df076d18d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:47.785726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5106efe5-b2e8-43eb-b2e7-6c3df076d18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:47.856179 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 454/914] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:24:47.879783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d533c-c3a4-49a4-bb40-d0cbcbc2e6c5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:47.974126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:47.988193 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d533c-c3a4-49a4-bb40-d0cbcbc2e6c5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:48.022978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94006b-a977-45e5-8817-d10443fb7b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:48.024132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94006b-a977-45e5-8817-d10443fb7b0b None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.024610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94006b-a977-45e5-8817-d10443fb7b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.025075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94006b-a977-45e5-8817-d10443fb7b0b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.035916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94006b-a977-45e5-8817-d10443fb7b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-bf94006b-a977-45e5-8817-d10443fb7b0b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:48.045191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:48.046148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:24:48.047091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:48.050505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 461/915] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jul 14 11:24:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:48.056603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106efe5-b2e8-43eb-b2e7-6c3df076d18d None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:48.085379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b3cfa-5627-4f37-9e16-9c92cdf22684 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.086003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b3cfa-5627-4f37-9e16-9c92cdf22684 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.086667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b3cfa-5627-4f37-9e16-9c92cdf22684 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.089093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 462/916] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:24:48.095117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ac7b8-ca5c-43d7-acc2-2536473716c6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.095396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ac7b8-ca5c-43d7-acc2-2536473716c6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.095668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ac7b8-ca5c-43d7-acc2-2536473716c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.115044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0b8ac7b8-ca5c-43d7-acc2-2536473716c6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:48.116226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0b8ac7b8-ca5c-43d7-acc2-2536473716c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:48.177264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106efe5-b2e8-43eb-b2e7-6c3df076d18d None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:48.179214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:48.193183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 463/917] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:48.215002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ac7b8-ca5c-43d7-acc2-2536473716c6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:48.231145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182a7bd-d962-4fe2-bc2d-1e3ee96843c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:48.232188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182a7bd-d962-4fe2-bc2d-1e3ee96843c2 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.232864 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182a7bd-d962-4fe2-bc2d-1e3ee96843c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.233407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182a7bd-d962-4fe2-bc2d-1e3ee96843c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.234500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182a7bd-d962-4fe2-bc2d-1e3ee96843c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:48.235040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182a7bd-d962-4fe2-bc2d-1e3ee96843c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:48.237849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182a7bd-d962-4fe2-bc2d-1e3ee96843c2 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:48.249149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 455/918] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jul 14 11:24:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:48.256812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182a7bd-d962-4fe2-bc2d-1e3ee96843c2 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:48.304374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:48.327208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ac7b8-ca5c-43d7-acc2-2536473716c6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:48.365787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff252d4-ec60-4fed-bfce-0f061b509992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:48.366768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff252d4-ec60-4fed-bfce-0f061b509992 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.367048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff252d4-ec60-4fed-bfce-0f061b509992 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.367317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff252d4-ec60-4fed-bfce-0f061b509992 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.367984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff252d4-ec60-4fed-bfce-0f061b509992 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:48.369425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff252d4-ec60-4fed-bfce-0f061b509992 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:48.370711 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182a7bd-d962-4fe2-bc2d-1e3ee96843c2 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-dff252d4-ec60-4fed-bfce-0f061b509992 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:48.382291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:48.383356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 11:24:48.384216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:48.388480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 464/919] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:24:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:48.389925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff252d4-ec60-4fed-bfce-0f061b509992 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:48.412131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0800c0d-bd36-4acb-8b8c-e4b202e12cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:48.413199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800c0d-bd36-4acb-8b8c-e4b202e12cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.413743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800c0d-bd36-4acb-8b8c-e4b202e12cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.414235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800c0d-bd36-4acb-8b8c-e4b202e12cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.427033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0800c0d-bd36-4acb-8b8c-e4b202e12cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:48.428999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0800c0d-bd36-4acb-8b8c-e4b202e12cb5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-d0800c0d-bd36-4acb-8b8c-e4b202e12cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:48.447916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:48.448861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:48.450078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:48.450078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:48.450078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:48.450078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:48.457007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 456/920] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:48.458514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0800c0d-bd36-4acb-8b8c-e4b202e12cb5 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:48.479041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:48.499084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff252d4-ec60-4fed-bfce-0f061b509992 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:48.534871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eab83-e437-4755-a0fc-e9a9c9d550be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:48.535437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eab83-e437-4755-a0fc-e9a9c9d550be None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.535735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eab83-e437-4755-a0fc-e9a9c9d550be None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.535982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eab83-e437-4755-a0fc-e9a9c9d550be None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.536683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365eab83-e437-4755-a0fc-e9a9c9d550be None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:48.536984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365eab83-e437-4755-a0fc-e9a9c9d550be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:48.538368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365eab83-e437-4755-a0fc-e9a9c9d550be None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:48.548410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 465/921] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:24:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:48.549419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eab83-e437-4755-a0fc-e9a9c9d550be None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:48.560208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:48.575317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0800c0d-bd36-4acb-8b8c-e4b202e12cb5 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:48.614514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7f575-eb92-475a-ae41-a03a8f3c9b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:48.615665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7f575-eb92-475a-ae41-a03a8f3c9b17 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.616275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7f575-eb92-475a-ae41-a03a8f3c9b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.616759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7f575-eb92-475a-ae41-a03a8f3c9b17 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.617945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7f575-eb92-475a-ae41-a03a8f3c9b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:48.618561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7f575-eb92-475a-ae41-a03a8f3c9b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:48.620265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7f575-eb92-475a-ae41-a03a8f3c9b17 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:48.633421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 457/922] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:48.640004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7f575-eb92-475a-ae41-a03a8f3c9b17 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:48.660263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eab83-e437-4755-a0fc-e9a9c9d550be None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:48.696538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fcbf2-d8f0-4a16-bd4e-fd84ac27e67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:48.697253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fcbf2-d8f0-4a16-bd4e-fd84ac27e67c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.697537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fcbf2-d8f0-4a16-bd4e-fd84ac27e67c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.698126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fcbf2-d8f0-4a16-bd4e-fd84ac27e67c None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.698616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fcbf2-d8f0-4a16-bd4e-fd84ac27e67c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:48.698920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fcbf2-d8f0-4a16-bd4e-fd84ac27e67c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:48.700625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fcbf2-d8f0-4a16-bd4e-fd84ac27e67c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:48.712092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 466/923] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jul 14 11:24:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:48.713321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fcbf2-d8f0-4a16-bd4e-fd84ac27e67c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:48.729929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:48.752112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7f575-eb92-475a-ae41-a03a8f3c9b17 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:48.790121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099d1f7-7dfc-4671-b8bd-eefa600dd006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:48.790791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:48.791885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099d1f7-7dfc-4671-b8bd-eefa600dd006 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.792235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099d1f7-7dfc-4671-b8bd-eefa600dd006 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.792521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099d1f7-7dfc-4671-b8bd-eefa600dd006 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.800901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099d1f7-7dfc-4671-b8bd-eefa600dd006 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:48.802350 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099d1f7-7dfc-4671-b8bd-eefa600dd006 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-f099d1f7-7dfc-4671-b8bd-eefa600dd006 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:48.817637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:48.818606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:48.819549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 458/924] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jul 14 11:24:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:48.820139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5383ec-e5b7-4596-a283-4e60dd77dcfa None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.820732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5383ec-e5b7-4596-a283-4e60dd77dcfa None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.820732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5383ec-e5b7-4596-a283-4e60dd77dcfa None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.822139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 459/925] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:48.826555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5383ec-e5b7-4596-a283-4e60dd77dcfa None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:48.830373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fcbf2-d8f0-4a16-bd4e-fd84ac27e67c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:48.873558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a68a9-31b9-442e-a560-1ce89900085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:48.874749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a68a9-31b9-442e-a560-1ce89900085f None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.875213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a68a9-31b9-442e-a560-1ce89900085f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.875492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a68a9-31b9-442e-a560-1ce89900085f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.876251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a68a9-31b9-442e-a560-1ce89900085f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:48.877664 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a68a9-31b9-442e-a560-1ce89900085f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:48.932796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5383ec-e5b7-4596-a283-4e60dd77dcfa None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:48.967401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101d33f-eb60-407f-bc76-bccb69cc0753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:48.968090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d33f-eb60-407f-bc76-bccb69cc0753 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.968376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d33f-eb60-407f-bc76-bccb69cc0753 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.968651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d33f-eb60-407f-bc76-bccb69cc0753 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:48.969530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101d33f-eb60-407f-bc76-bccb69cc0753 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:48.969842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101d33f-eb60-407f-bc76-bccb69cc0753 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:48.971276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101d33f-eb60-407f-bc76-bccb69cc0753 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:48.973932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:48.988319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 460/926] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jul 14 11:24:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:48.989570 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084f9bd-385f-4893-8779-66687d1f05c8 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:48.989861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084f9bd-385f-4893-8779-66687d1f05c8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:48.990139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084f9bd-385f-4893-8779-66687d1f05c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.006828 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e084f9bd-385f-4893-8779-66687d1f05c8 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:49.008025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e084f9bd-385f-4893-8779-66687d1f05c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:49.024571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 467/927] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jul 14 11:24:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:49.026128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a68a9-31b9-442e-a560-1ce89900085f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:49.105711 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 461/928] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:49.123805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084f9bd-385f-4893-8779-66687d1f05c8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:49.173190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a68a9-31b9-442e-a560-1ce89900085f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:49.206527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:49.215543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080ffb3-550f-48b1-a998-ade9d96f7031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:49.216443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080ffb3-550f-48b1-a998-ade9d96f7031 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:49.216723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080ffb3-550f-48b1-a998-ade9d96f7031 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:49.217016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080ffb3-550f-48b1-a998-ade9d96f7031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.229121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080ffb3-550f-48b1-a998-ade9d96f7031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:49.230620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080ffb3-550f-48b1-a998-ade9d96f7031 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:49.239340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084f9bd-385f-4893-8779-66687d1f05c8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-a080ffb3-550f-48b1-a998-ade9d96f7031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:49.251363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:49.252361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:49.254109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:49.254109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:49.254109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:49.254109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:49.254109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 468/929] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:49.254756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94acbe7-c826-423e-9534-6b8cda86dfac None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:49.255045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94acbe7-c826-423e-9534-6b8cda86dfac None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:49.255322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94acbe7-c826-423e-9534-6b8cda86dfac None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.256598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 469/930] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:49.262599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf867c1-6fdd-4e5f-ac0a-016026c100dd None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:49.263303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf867c1-6fdd-4e5f-ac0a-016026c100dd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:49.263303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf867c1-6fdd-4e5f-ac0a-016026c100dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.265031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:49.280668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1cf867c1-6fdd-4e5f-ac0a-016026c100dd None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:49.281995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1cf867c1-6fdd-4e5f-ac0a-016026c100dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:49.282893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60461fac-5f22-43c2-959f-ca664455e698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:49.283934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60461fac-5f22-43c2-959f-ca664455e698 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:49.284315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60461fac-5f22-43c2-959f-ca664455e698 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:49.284592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60461fac-5f22-43c2-959f-ca664455e698 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.285597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60461fac-5f22-43c2-959f-ca664455e698 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:49.286644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60461fac-5f22-43c2-959f-ca664455e698 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-60461fac-5f22-43c2-959f-ca664455e698 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:49.298932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:49.300029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:49.300885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 462/931] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:49.301760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60461fac-5f22-43c2-959f-ca664455e698 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:49.364143 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 470/932] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:49.364810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf867c1-6fdd-4e5f-ac0a-016026c100dd None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:49.425712 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60461fac-5f22-43c2-959f-ca664455e698 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:49.468726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fa3bc-2499-484c-81c1-6f938cbcd660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:49.469903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fa3bc-2499-484c-81c1-6f938cbcd660 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:49.470407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fa3bc-2499-484c-81c1-6f938cbcd660 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:49.470791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fa3bc-2499-484c-81c1-6f938cbcd660 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.471583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fa3bc-2499-484c-81c1-6f938cbcd660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:49.471978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fa3bc-2499-484c-81c1-6f938cbcd660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:49.473513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fa3bc-2499-484c-81c1-6f938cbcd660 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:49.489044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 463/933] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:49.489923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fa3bc-2499-484c-81c1-6f938cbcd660 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:49.490552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf867c1-6fdd-4e5f-ac0a-016026c100dd None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:49.504301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:49.530729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283aa7f-5553-42e9-8b9f-4266077d89f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:49.532611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283aa7f-5553-42e9-8b9f-4266077d89f7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:49.533078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283aa7f-5553-42e9-8b9f-4266077d89f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:49.533622 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283aa7f-5553-42e9-8b9f-4266077d89f7 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.534534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283aa7f-5553-42e9-8b9f-4266077d89f7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:49.534991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283aa7f-5553-42e9-8b9f-4266077d89f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:49.536518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283aa7f-5553-42e9-8b9f-4266077d89f7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:49.547076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 471/934] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jul 14 11:24:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:49.547076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283aa7f-5553-42e9-8b9f-4266077d89f7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:49.548205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:49.616184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fa3bc-2499-484c-81c1-6f938cbcd660 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:49.656826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fee606-dc65-4858-b15d-28b2f96e0e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:49.657666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fee606-dc65-4858-b15d-28b2f96e0e6d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:49.657947 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fee606-dc65-4858-b15d-28b2f96e0e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:49.658219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fee606-dc65-4858-b15d-28b2f96e0e6d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.658927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fee606-dc65-4858-b15d-28b2f96e0e6d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:49.660915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fee606-dc65-4858-b15d-28b2f96e0e6d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:49.670878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283aa7f-5553-42e9-8b9f-4266077d89f7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-c4fee606-dc65-4858-b15d-28b2f96e0e6d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:49.673744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:49.674731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:49.675613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 464/935] 10.222.0.40 () {56 vars in 1132 bytes} [Thu Jul 14 11:24:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:49.676481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fee606-dc65-4858-b15d-28b2f96e0e6d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:49.712088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcde51-2e7c-4d83-960e-5ef48a284b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:49.713296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcde51-2e7c-4d83-960e-5ef48a284b44 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:49.713811 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcde51-2e7c-4d83-960e-5ef48a284b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:49.714236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcde51-2e7c-4d83-960e-5ef48a284b44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.721825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcde51-2e7c-4d83-960e-5ef48a284b44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:49.723469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcde51-2e7c-4d83-960e-5ef48a284b44 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4bbcde51-2e7c-4d83-960e-5ef48a284b44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:49.737738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:49.738857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:49.739966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:49.739966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:49.739966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:49.739966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:49.739966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:49.739966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:49.739966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:49.739966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:49.741806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 472/936] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:49.742758 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcde51-2e7c-4d83-960e-5ef48a284b44 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:49.778982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:49.790732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fee606-dc65-4858-b15d-28b2f96e0e6d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:49.801364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:49.826579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739de55-006a-4c9e-9bd8-373e712e33a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:49.827251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739de55-006a-4c9e-9bd8-373e712e33a6 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:49.827534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739de55-006a-4c9e-9bd8-373e712e33a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:49.827808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739de55-006a-4c9e-9bd8-373e712e33a6 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.828527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739de55-006a-4c9e-9bd8-373e712e33a6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:49.828832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739de55-006a-4c9e-9bd8-373e712e33a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:49.830245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739de55-006a-4c9e-9bd8-373e712e33a6 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:49.836483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 465/937] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jul 14 11:24:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:49.837125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739de55-006a-4c9e-9bd8-373e712e33a6 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:49.861306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcde51-2e7c-4d83-960e-5ef48a284b44 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:49.901422 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9327d12-44fc-47b6-9129-29309a1c635a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:49.902009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9327d12-44fc-47b6-9129-29309a1c635a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:49.902294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9327d12-44fc-47b6-9129-29309a1c635a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:49.902572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9327d12-44fc-47b6-9129-29309a1c635a None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:49.903340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9327d12-44fc-47b6-9129-29309a1c635a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:49.903656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9327d12-44fc-47b6-9129-29309a1c635a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:49.905176 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9327d12-44fc-47b6-9129-29309a1c635a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:49.911614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 473/938] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:49.912560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9327d12-44fc-47b6-9129-29309a1c635a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:49.957188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739de55-006a-4c9e-9bd8-373e712e33a6 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:50.003414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14093c-8de3-468c-8a55-e525a2df1357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:50.004081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14093c-8de3-468c-8a55-e525a2df1357 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:50.004361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14093c-8de3-468c-8a55-e525a2df1357 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:50.004631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14093c-8de3-468c-8a55-e525a2df1357 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:50.005464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14093c-8de3-468c-8a55-e525a2df1357 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:50.005766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14093c-8de3-468c-8a55-e525a2df1357 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:50.007201 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14093c-8de3-468c-8a55-e525a2df1357 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:50.025589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 466/939] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:50.025589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14093c-8de3-468c-8a55-e525a2df1357 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:50.033255 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:50.039987 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9327d12-44fc-47b6-9129-29309a1c635a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:50.058099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:50.075972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9547-bfb4-43ce-af76-0f52930f7bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:50.081660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9547-bfb4-43ce-af76-0f52930f7bfe None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:50.081660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9547-bfb4-43ce-af76-0f52930f7bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:50.081660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9547-bfb4-43ce-af76-0f52930f7bfe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:50.081660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a9547-bfb4-43ce-af76-0f52930f7bfe None admin] RBAC: Adding query filter params (interface=public, service_id=3f6b262c75e94595aee6aec821166499, region_id=RegionOne) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:50.081660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a9547-bfb4-43ce-af76-0f52930f7bfe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:50.081660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a9547-bfb4-43ce-af76-0f52930f7bfe None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:50.085544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 474/940] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jul 14 11:24:49 2022] GET /identity/v3/endpoints?service_id=3f6b262c75e94595aee6aec821166499&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:50.086132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9547-bfb4-43ce-af76-0f52930f7bfe None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:50.135629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14093c-8de3-468c-8a55-e525a2df1357 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:50.172122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d47516-0502-4604-8c61-ff817d1e7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:50.172706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d47516-0502-4604-8c61-ff817d1e7f72 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:50.172990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d47516-0502-4604-8c61-ff817d1e7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:50.173303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d47516-0502-4604-8c61-ff817d1e7f72 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:50.174017 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d47516-0502-4604-8c61-ff817d1e7f72 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:50.174325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d47516-0502-4604-8c61-ff817d1e7f72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:50.176240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d47516-0502-4604-8c61-ff817d1e7f72 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:50.181288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 467/941] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:50.182091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d47516-0502-4604-8c61-ff817d1e7f72 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:50.195884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9547-bfb4-43ce-af76-0f52930f7bfe None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:50.234158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae61e5-7f4e-48a9-bf25-284a2a42ec4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:50.235047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae61e5-7f4e-48a9-bf25-284a2a42ec4a None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:50.235330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae61e5-7f4e-48a9-bf25-284a2a42ec4a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:50.235602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae61e5-7f4e-48a9-bf25-284a2a42ec4a None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/0bc624eba860448eab54b516bd344a46/roles/b7e771a1c5f54085a2a9174024f86f19` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:50.238399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae61e5-7f4e-48a9-bf25-284a2a42ec4a None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=0bc624eba860448eab54b516bd344a46, role_id=b7e771a1c5f54085a2a9174024f86f19)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:50.239848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae61e5-7f4e-48a9-bf25-284a2a42ec4a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:50.282456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:50.288106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d47516-0502-4604-8c61-ff817d1e7f72 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:50.321923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3966ffa-40c0-4125-b815-6a54d26ccca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:50.322738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3966ffa-40c0-4125-b815-6a54d26ccca4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:50.323018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3966ffa-40c0-4125-b815-6a54d26ccca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:50.323289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3966ffa-40c0-4125-b815-6a54d26ccca4 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:50.323993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3966ffa-40c0-4125-b815-6a54d26ccca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:50.325687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3966ffa-40c0-4125-b815-6a54d26ccca4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:50.332117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 468/942] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jul 14 11:24:50 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:24:50.333294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3966ffa-40c0-4125-b815-6a54d26ccca4 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:50.370547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 475/943] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:24:50 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/0bc624eba860448eab54b516bd344a46/roles/b7e771a1c5f54085a2a9174024f86f19 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:50.475981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3966ffa-40c0-4125-b815-6a54d26ccca4 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:50.517940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098ff6c-83b8-4557-bf39-6b698b61dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:50.518687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ff6c-83b8-4557-bf39-6b698b61dc78 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:50.518973 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ff6c-83b8-4557-bf39-6b698b61dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:50.519250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ff6c-83b8-4557-bf39-6b698b61dc78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:50.520042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098ff6c-83b8-4557-bf39-6b698b61dc78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:50.521605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098ff6c-83b8-4557-bf39-6b698b61dc78 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:50.650498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 469/944] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jul 14 11:24:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:51.144126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c01d2-62fc-4507-8783-e12b6aa1c717 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:51.144713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c01d2-62fc-4507-8783-e12b6aa1c717 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:51.144713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c01d2-62fc-4507-8783-e12b6aa1c717 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:51.146265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 476/945] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:51.152016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362a04b-4f7b-4024-80be-539e778b5b81 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:51.152547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362a04b-4f7b-4024-80be-539e778b5b81 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:51.152547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362a04b-4f7b-4024-80be-539e778b5b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:51.170294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-3362a04b-4f7b-4024-80be-539e778b5b81 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:51.171477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3362a04b-4f7b-4024-80be-539e778b5b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:51.183265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:51.244519 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 470/946] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:51.267929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c01d2-62fc-4507-8783-e12b6aa1c717 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:51.291991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:51.373829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c01d2-62fc-4507-8783-e12b6aa1c717 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:51.408268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:51.409475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:51.409792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:51.410072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:51.410798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:51.412241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:51.413661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WARNING keystone.server.flask.common [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 11:24:51.466299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.sql.core [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 11:24:51.466299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 11:24:51.466299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [parameters: {'id': '66d8ab56987342eca16c47d3d702381d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 11:24:51.466299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80433) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66d8ab56987342eca16c47d3d702381d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66d8ab56987342eca16c47d3d702381d. Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application conn.query(q) Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 11:24:51.475362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result.read() Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 11:24:51.476228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application next(self.gen) Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application yield resource Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application next(self.gen) Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise self.value Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application session.commit() Jul 14 11:24:51.477295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.session.flush() Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application compat.raise_( Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 11:24:51.478213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 11:24:51.479128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application conn.query(q) Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result.read() Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 11:24:51.479928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [parameters: {'id': '66d8ab56987342eca16c47d3d702381d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:51.480794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:51.483331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:51.485584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:51.485584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 11:24:51.485584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 11:24:51.485584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:51.485584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:51.485584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 11:24:51.485584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 11:24:51.485584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66d8ab56987342eca16c47d3d702381d. Jul 14 11:24:51.485584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:51.489560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 477/947] 10.222.0.40 () {60 vars in 1140 bytes} [Thu Jul 14 11:24:51 2022] POST /identity/v3/projects => generated 262 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:51.495547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362a04b-4f7b-4024-80be-539e778b5b81 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:51.603736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362a04b-4f7b-4024-80be-539e778b5b81 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:51.638636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b38cd-9b17-4558-80e2-1ddddbbdac8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:51.639426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b38cd-9b17-4558-80e2-1ddddbbdac8e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:51.639712 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b38cd-9b17-4558-80e2-1ddddbbdac8e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:51.639990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b38cd-9b17-4558-80e2-1ddddbbdac8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:51.647438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b38cd-9b17-4558-80e2-1ddddbbdac8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:51.648898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b38cd-9b17-4558-80e2-1ddddbbdac8e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-177b38cd-9b17-4558-80e2-1ddddbbdac8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:51.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:51.665309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:51.666272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:51.666272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:51.666272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:51.666272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:51.666272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:51.666272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:51.666272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:51.666272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:51.666272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 471/948] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:51.670582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:51.750614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:51.773132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9967c-f937-4262-843a-89fde69618c9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:51.812494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84cb01-6d13-41af-8146-eba7873845b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:51.813184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84cb01-6d13-41af-8146-eba7873845b8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:51.813475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84cb01-6d13-41af-8146-eba7873845b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:51.813749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84cb01-6d13-41af-8146-eba7873845b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:51.814519 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84cb01-6d13-41af-8146-eba7873845b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:51.814825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84cb01-6d13-41af-8146-eba7873845b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:51.816488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84cb01-6d13-41af-8146-eba7873845b8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:51.839021 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 478/949] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:52.500640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda24a09-4656-4225-bca3-ebc499fc2948 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.501209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda24a09-4656-4225-bca3-ebc499fc2948 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.501209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda24a09-4656-4225-bca3-ebc499fc2948 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.503493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 472/950] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:52.511816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9c22c-f39d-4adc-9d7d-31d40a95fba1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.512337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9c22c-f39d-4adc-9d7d-31d40a95fba1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.512433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9c22c-f39d-4adc-9d7d-31d40a95fba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.533997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-11a9c22c-f39d-4adc-9d7d-31d40a95fba1 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:52.535512 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-11a9c22c-f39d-4adc-9d7d-31d40a95fba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:52.568223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccafeb8-9dac-4b57-b19c-ffc54c9f0300 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.568849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccafeb8-9dac-4b57-b19c-ffc54c9f0300 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.569342 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccafeb8-9dac-4b57-b19c-ffc54c9f0300 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.571597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 473/951] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:24:52.577151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3030fc5-56ce-425e-95d1-172a8c658a16 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.577762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3030fc5-56ce-425e-95d1-172a8c658a16 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.578249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3030fc5-56ce-425e-95d1-172a8c658a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.584368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:52.597431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-a3030fc5-56ce-425e-95d1-172a8c658a16 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:52.598624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a3030fc5-56ce-425e-95d1-172a8c658a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:52.616228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 479/952] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:52.640855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9c22c-f39d-4adc-9d7d-31d40a95fba1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:52.689391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 474/953] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:24:52.706835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3030fc5-56ce-425e-95d1-172a8c658a16 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:52.720277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:52.758353 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9c22c-f39d-4adc-9d7d-31d40a95fba1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:52.797526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e51df7-8718-4c64-8a70-5a5e333d151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:52.798736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e51df7-8718-4c64-8a70-5a5e333d151d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.799221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e51df7-8718-4c64-8a70-5a5e333d151d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.800306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e51df7-8718-4c64-8a70-5a5e333d151d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.801215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e51df7-8718-4c64-8a70-5a5e333d151d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:52.803085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e51df7-8718-4c64-8a70-5a5e333d151d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-80e51df7-8718-4c64-8a70-5a5e333d151d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:52.817370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:52.818528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:52.819427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:52.825187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 480/954] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:52.825187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bb002-7691-4421-a2e1-a7311298d2c3 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.825187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bb002-7691-4421-a2e1-a7311298d2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.825187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bb002-7691-4421-a2e1-a7311298d2c3 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.827612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 481/955] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:52.827745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3030fc5-56ce-425e-95d1-172a8c658a16 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:52.830899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bb002-7691-4421-a2e1-a7311298d2c3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:52.868243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:52.872728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471eb87-220a-4675-801b-fffcc6b4142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:52.873441 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471eb87-220a-4675-801b-fffcc6b4142a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.874119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471eb87-220a-4675-801b-fffcc6b4142a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.874119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471eb87-220a-4675-801b-fffcc6b4142a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.885377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471eb87-220a-4675-801b-fffcc6b4142a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-5471eb87-220a-4675-801b-fffcc6b4142a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:24:52.894207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:52.895173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:52.896566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 475/956] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jul 14 11:24:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:52.897532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96616278-2a25-4fb4-a3ee-99767a5fc438 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.897532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96616278-2a25-4fb4-a3ee-99767a5fc438 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.897532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96616278-2a25-4fb4-a3ee-99767a5fc438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.915526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-96616278-2a25-4fb4-a3ee-99767a5fc438 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:52.916735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-96616278-2a25-4fb4-a3ee-99767a5fc438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:52.941481 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bb002-7691-4421-a2e1-a7311298d2c3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:52.972466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:52.979360 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fbe32-6ade-4a22-b2b3-dfea070d7bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:52.980181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fbe32-6ade-4a22-b2b3-dfea070d7bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.980775 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fbe32-6ade-4a22-b2b3-dfea070d7bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.981058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fbe32-6ade-4a22-b2b3-dfea070d7bc5 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.981958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fbe32-6ade-4a22-b2b3-dfea070d7bc5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:52.982264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fbe32-6ade-4a22-b2b3-dfea070d7bc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:52.983626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fbe32-6ade-4a22-b2b3-dfea070d7bc5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:52.997445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 476/957] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:52.997445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96616278-2a25-4fb4-a3ee-99767a5fc438 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:52.997782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 482/958] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jul 14 11:24:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:52.997782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fbe32-6ade-4a22-b2b3-dfea070d7bc5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.106685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96616278-2a25-4fb4-a3ee-99767a5fc438 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.107513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fbe32-6ade-4a22-b2b3-dfea070d7bc5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.130436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:53.142985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be85fe6-42a6-4e3e-8afb-077eccd2a73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.143670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49712bda-ebd5-4f72-95ef-ec9770332277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.144314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be85fe6-42a6-4e3e-8afb-077eccd2a73f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.144594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be85fe6-42a6-4e3e-8afb-077eccd2a73f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.144866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be85fe6-42a6-4e3e-8afb-077eccd2a73f None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.144988 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49712bda-ebd5-4f72-95ef-ec9770332277 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.145432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49712bda-ebd5-4f72-95ef-ec9770332277 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.145649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be85fe6-42a6-4e3e-8afb-077eccd2a73f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:53.145952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be85fe6-42a6-4e3e-8afb-077eccd2a73f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.146145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49712bda-ebd5-4f72-95ef-ec9770332277 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.146990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49712bda-ebd5-4f72-95ef-ec9770332277 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:53.147331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be85fe6-42a6-4e3e-8afb-077eccd2a73f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.147584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49712bda-ebd5-4f72-95ef-ec9770332277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.149087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49712bda-ebd5-4f72-95ef-ec9770332277 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.154851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 483/959] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jul 14 11:24:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:53.155903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49712bda-ebd5-4f72-95ef-ec9770332277 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.160600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 477/960] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jul 14 11:24:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:53.161488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be85fe6-42a6-4e3e-8afb-077eccd2a73f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.218325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:53.271048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49712bda-ebd5-4f72-95ef-ec9770332277 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.280179 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be85fe6-42a6-4e3e-8afb-077eccd2a73f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.307980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea40ac-60c3-41ce-8b8e-55f4d38e35c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.308790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea40ac-60c3-41ce-8b8e-55f4d38e35c7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.309069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea40ac-60c3-41ce-8b8e-55f4d38e35c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.309370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea40ac-60c3-41ce-8b8e-55f4d38e35c7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.310017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea40ac-60c3-41ce-8b8e-55f4d38e35c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.311377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea40ac-60c3-41ce-8b8e-55f4d38e35c7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.318716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a05079-19b1-4d0d-9b68-757950b3df7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.319580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a05079-19b1-4d0d-9b68-757950b3df7c None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.320058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a05079-19b1-4d0d-9b68-757950b3df7c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.320511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a05079-19b1-4d0d-9b68-757950b3df7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.321506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a05079-19b1-4d0d-9b68-757950b3df7c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.323149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a05079-19b1-4d0d-9b68-757950b3df7c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-7cea40ac-60c3-41ce-8b8e-55f4d38e35c7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:53.323454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:53.324729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:53.325600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 484/961] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:53.326397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea40ac-60c3-41ce-8b8e-55f4d38e35c7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.425268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 478/962] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jul 14 11:24:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:53.426467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a05079-19b1-4d0d-9b68-757950b3df7c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.448199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea40ac-60c3-41ce-8b8e-55f4d38e35c7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.482387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:53.493802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:53.537388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261b6e8-367a-429a-ad4d-c1c805662103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.538596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261b6e8-367a-429a-ad4d-c1c805662103 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.538875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261b6e8-367a-429a-ad4d-c1c805662103 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.539144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261b6e8-367a-429a-ad4d-c1c805662103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.554212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261b6e8-367a-429a-ad4d-c1c805662103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.554620 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a05079-19b1-4d0d-9b68-757950b3df7c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.555695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261b6e8-367a-429a-ad4d-c1c805662103 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-6261b6e8-367a-429a-ad4d-c1c805662103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:53.579274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:53.580334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:53.581945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:53.581945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:53.581945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:53.581945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:53.581945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 485/963] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:53.588721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261b6e8-367a-429a-ad4d-c1c805662103 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.602176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56399aa1-b8b9-4bb1-adf0-7bf0377ec4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.603931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56399aa1-b8b9-4bb1-adf0-7bf0377ec4ab None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.604435 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56399aa1-b8b9-4bb1-adf0-7bf0377ec4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.604941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56399aa1-b8b9-4bb1-adf0-7bf0377ec4ab None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.605914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56399aa1-b8b9-4bb1-adf0-7bf0377ec4ab None admin] RBAC: Adding query filter params (name=compute) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:53.606464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56399aa1-b8b9-4bb1-adf0-7bf0377ec4ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.608313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56399aa1-b8b9-4bb1-adf0-7bf0377ec4ab None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.614424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 479/964] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:53.622042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56399aa1-b8b9-4bb1-adf0-7bf0377ec4ab None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.712423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261b6e8-367a-429a-ad4d-c1c805662103 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.750072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ca7c7-7451-42a3-a4ba-a88400e3df12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.750739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca7c7-7451-42a3-a4ba-a88400e3df12 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.751024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca7c7-7451-42a3-a4ba-a88400e3df12 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.751301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca7c7-7451-42a3-a4ba-a88400e3df12 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.752054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ca7c7-7451-42a3-a4ba-a88400e3df12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:53.752331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ca7c7-7451-42a3-a4ba-a88400e3df12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.754356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ca7c7-7451-42a3-a4ba-a88400e3df12 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.769091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56399aa1-b8b9-4bb1-adf0-7bf0377ec4ab None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.769331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 486/965] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:53.775520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ca7c7-7451-42a3-a4ba-a88400e3df12 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.776931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:53.816407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:53.817810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd3f80-d07b-4670-9cae-efbe24577385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.818752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd3f80-d07b-4670-9cae-efbe24577385 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.819266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd3f80-d07b-4670-9cae-efbe24577385 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.819782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd3f80-d07b-4670-9cae-efbe24577385 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.820711 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd3f80-d07b-4670-9cae-efbe24577385 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:53.821248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd3f80-d07b-4670-9cae-efbe24577385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.822938 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd3f80-d07b-4670-9cae-efbe24577385 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.828802 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 480/966] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:53.897566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ca7c7-7451-42a3-a4ba-a88400e3df12 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.937788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9c563-aaf2-4e0e-9a57-c8969bd5e062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.938635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9c563-aaf2-4e0e-9a57-c8969bd5e062 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.938914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9c563-aaf2-4e0e-9a57-c8969bd5e062 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.939191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9c563-aaf2-4e0e-9a57-c8969bd5e062 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.948530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9c563-aaf2-4e0e-9a57-c8969bd5e062 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.950315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9c563-aaf2-4e0e-9a57-c8969bd5e062 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.964038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee612e9-3d4b-4328-a7cc-cc7ef326ea86 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.964229 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee612e9-3d4b-4328-a7cc-cc7ef326ea86 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.964604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee612e9-3d4b-4328-a7cc-cc7ef326ea86 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-0de9c563-aaf2-4e0e-9a57-c8969bd5e062 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:53.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:53.968788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:53.969862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 487/968] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jul 14 11:24:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:53.971083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 481/968] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:24:53.974858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff73a-e793-401f-8bd8-2ef31a2eae40 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.975150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff73a-e793-401f-8bd8-2ef31a2eae40 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.975328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee612e9-3d4b-4328-a7cc-cc7ef326ea86 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.975488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff73a-e793-401f-8bd8-2ef31a2eae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.993970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-88aff73a-e793-401f-8bd8-2ef31a2eae40 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:53.995162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-88aff73a-e793-401f-8bd8-2ef31a2eae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:54.093199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 488/969] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:54.122046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff73a-e793-401f-8bd8-2ef31a2eae40 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:54.131542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee612e9-3d4b-4328-a7cc-cc7ef326ea86 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:54.146753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:54.167651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23969bb2-f584-48cc-898a-edf91e1b2b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:54.168015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23969bb2-f584-48cc-898a-edf91e1b2b7f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:54.168239 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23969bb2-f584-48cc-898a-edf91e1b2b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:54.168510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23969bb2-f584-48cc-898a-edf91e1b2b7f None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:54.169463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23969bb2-f584-48cc-898a-edf91e1b2b7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:54.169773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23969bb2-f584-48cc-898a-edf91e1b2b7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:54.171261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23969bb2-f584-48cc-898a-edf91e1b2b7f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:54.191198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 482/970] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jul 14 11:24:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:54.196217 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23969bb2-f584-48cc-898a-edf91e1b2b7f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:54.225632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:54.249270 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff73a-e793-401f-8bd8-2ef31a2eae40 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:54.287336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46741923-a390-4434-80ee-c375f71b6283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:54.288029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46741923-a390-4434-80ee-c375f71b6283 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:54.288312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46741923-a390-4434-80ee-c375f71b6283 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:54.288588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46741923-a390-4434-80ee-c375f71b6283 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:54.289331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46741923-a390-4434-80ee-c375f71b6283 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:54.290871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46741923-a390-4434-80ee-c375f71b6283 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:54.315995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23969bb2-f584-48cc-898a-edf91e1b2b7f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:54.346045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.sql.core [None req-46741923-a390-4434-80ee-c375f71b6283 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 11:24:54.346045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 11:24:54.346045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [parameters: {'user_id': 'b9ccd3d009924944a5c7c73e126d7279', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 11:24:54.346045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80433) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-46741923-a390-4434-80ee-c375f71b6283 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application conn.query(q) Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result.read() Jul 14 11:24:54.355084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 11:24:54.355974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application next(self.gen) Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application yield resource Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application next(self.gen) Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise self.value Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application session.commit() Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 11:24:54.356944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.session.flush() Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application compat.raise_( Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 11:24:54.357888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 11:24:54.358787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application conn.query(q) Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application result.read() Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 11:24:54.360587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b9ccd3d009924944a5c7c73e126d7279', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:54.363174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 11:24:54.365063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 11:24:54.368689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 11:24:54.368689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 11:24:54.368689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 11:24:54.368689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 11:24:54.368689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 11:24:54.368689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:54.369508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17dfcd-58da-4b2f-b771-4062d5922296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:54.369508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17dfcd-58da-4b2f-b771-4062d5922296 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:54.369508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17dfcd-58da-4b2f-b771-4062d5922296 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:54.369508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17dfcd-58da-4b2f-b771-4062d5922296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:54.370388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 489/971] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jul 14 11:24:54 2022] POST /identity/v3/users => generated 160 bytes in 249 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:54.375177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46741923-a390-4434-80ee-c375f71b6283 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:54.436406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17dfcd-58da-4b2f-b771-4062d5922296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:54.437901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f17dfcd-58da-4b2f-b771-4062d5922296 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:54.465037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:54.468961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:54.488970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46741923-a390-4434-80ee-c375f71b6283 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:54.525440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25392631-7bf2-4d4d-a0ce-82a1d4c6155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:54.526247 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25392631-7bf2-4d4d-a0ce-82a1d4c6155f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:54.526527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25392631-7bf2-4d4d-a0ce-82a1d4c6155f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:54.526802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25392631-7bf2-4d4d-a0ce-82a1d4c6155f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:54.537089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25392631-7bf2-4d4d-a0ce-82a1d4c6155f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:54.539081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25392631-7bf2-4d4d-a0ce-82a1d4c6155f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:54.544396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 483/972] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:24:54 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 11:24:54.551253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17dfcd-58da-4b2f-b771-4062d5922296 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-25392631-7bf2-4d4d-a0ce-82a1d4c6155f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:24:54.561146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:54.562116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:54.563136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:54.563136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:54.563136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:24:54.563136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:24:54.563136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:24:54.563136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:24:54.563136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:24:54.563136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:24:54.563136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 11:24:54.563136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:54.566410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 490/973] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:24:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:54.575385 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25392631-7bf2-4d4d-a0ce-82a1d4c6155f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:54.671112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17dfcd-58da-4b2f-b771-4062d5922296 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:54.698604 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25392631-7bf2-4d4d-a0ce-82a1d4c6155f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:54.707544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:54.719708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda4720-f3c0-4376-ab8c-848ae7f53aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:54.721309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda4720-f3c0-4376-ab8c-848ae7f53aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:54.725410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda4720-f3c0-4376-ab8c-848ae7f53aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:54.725410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda4720-f3c0-4376-ab8c-848ae7f53aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:54.735521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618f2b6-fba5-448e-864a-f3b146b32b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:54.736394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f2b6-fba5-448e-864a-f3b146b32b30 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:54.737317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f2b6-fba5-448e-864a-f3b146b32b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:54.737694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f2b6-fba5-448e-864a-f3b146b32b30 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:54.738219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda4720-f3c0-4376-ab8c-848ae7f53aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:54.738700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618f2b6-fba5-448e-864a-f3b146b32b30 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:54.739160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618f2b6-fba5-448e-864a-f3b146b32b30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:54.740245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda4720-f3c0-4376-ab8c-848ae7f53aa2 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:54.740708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618f2b6-fba5-448e-864a-f3b146b32b30 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-fcda4720-f3c0-4376-ab8c-848ae7f53aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:24:54.760432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:54.761531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:54.762581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:24:54.762581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:24:54.762581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:24:54.762581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:54.766788 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 484/974] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:54.767763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda4720-f3c0-4376-ab8c-848ae7f53aa2 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:54.786147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:54.801448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 491/975] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jul 14 11:24:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:54.894035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda4720-f3c0-4376-ab8c-848ae7f53aa2 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:54.936638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c53325-df10-4b86-afd8-5fac037609fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:54.937719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c53325-df10-4b86-afd8-5fac037609fd None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:54.938194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c53325-df10-4b86-afd8-5fac037609fd None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:54.938655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c53325-df10-4b86-afd8-5fac037609fd None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:54.939596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c53325-df10-4b86-afd8-5fac037609fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:54.940683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c53325-df10-4b86-afd8-5fac037609fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:54.942398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c53325-df10-4b86-afd8-5fac037609fd None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:54.959906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 485/976] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:54.963346 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618f2b6-fba5-448e-864a-f3b146b32b30 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:55.078170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618f2b6-fba5-448e-864a-f3b146b32b30 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:55.115665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98cefd-f001-4dc3-89d1-b1b621bc447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:55.116843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98cefd-f001-4dc3-89d1-b1b621bc447a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:55.118043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98cefd-f001-4dc3-89d1-b1b621bc447a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:55.119566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98cefd-f001-4dc3-89d1-b1b621bc447a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:55.126580 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98cefd-f001-4dc3-89d1-b1b621bc447a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:55.128101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98cefd-f001-4dc3-89d1-b1b621bc447a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-aa98cefd-f001-4dc3-89d1-b1b621bc447a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:24:55.142351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:55.143304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:55.148859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:24:55.148859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:24:55.148859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:24:55.148859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:24:55.148859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:24:55.148859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:24:55.148859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:24:55.148859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:55.148859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 492/977] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:55.150986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c53325-df10-4b86-afd8-5fac037609fd None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:55.193743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:55.253767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c53325-df10-4b86-afd8-5fac037609fd None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:55.287815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb823-1265-4e5a-822f-b1f3b6de9c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:55.288470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb823-1265-4e5a-822f-b1f3b6de9c98 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:55.288762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb823-1265-4e5a-822f-b1f3b6de9c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:55.289033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb823-1265-4e5a-822f-b1f3b6de9c98 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:55.289852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3eb823-1265-4e5a-822f-b1f3b6de9c98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:55.290159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3eb823-1265-4e5a-822f-b1f3b6de9c98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:55.291644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3eb823-1265-4e5a-822f-b1f3b6de9c98 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:55.303970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 486/978] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:24:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:55.309933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98cefd-f001-4dc3-89d1-b1b621bc447a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:55.371916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:55.417798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98cefd-f001-4dc3-89d1-b1b621bc447a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:55.451665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f32562-1c75-4558-b89f-fbde3121f807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:55.452317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f32562-1c75-4558-b89f-fbde3121f807 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:55.452598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f32562-1c75-4558-b89f-fbde3121f807 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:55.452868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f32562-1c75-4558-b89f-fbde3121f807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:55.453741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f32562-1c75-4558-b89f-fbde3121f807 None admin] RBAC: Adding query filter params (role.id=b7e771a1c5f54085a2a9174024f86f19, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=0bc624eba860448eab54b516bd344a46) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:55.454052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f32562-1c75-4558-b89f-fbde3121f807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:55.456010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f32562-1c75-4558-b89f-fbde3121f807 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:55.469188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 493/979] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:24:55 2022] GET /identity/v3/role_assignments?user.id=0bc624eba860448eab54b516bd344a46&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=b7e771a1c5f54085a2a9174024f86f19 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:55.567903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644346e4-0cdf-4caa-945d-5a85451396a5 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:55.568195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644346e4-0cdf-4caa-945d-5a85451396a5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:55.568481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644346e4-0cdf-4caa-945d-5a85451396a5 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:55.569900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 487/980] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:55.575848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f6a90-5a41-4d84-8d5a-e478b64f0dab None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:55.576383 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f6a90-5a41-4d84-8d5a-e478b64f0dab None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:55.576383 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f6a90-5a41-4d84-8d5a-e478b64f0dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:55.594347 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4a0f6a90-5a41-4d84-8d5a-e478b64f0dab None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:55.595901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4a0f6a90-5a41-4d84-8d5a-e478b64f0dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:55.668887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 494/981] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:55.692703 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644346e4-0cdf-4caa-945d-5a85451396a5 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:55.791617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:55.800170 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644346e4-0cdf-4caa-945d-5a85451396a5 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:55.835058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8f239-b03a-4d0d-9f88-08bfc4e0dd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:55.835866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8f239-b03a-4d0d-9f88-08bfc4e0dd0e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:55.836151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8f239-b03a-4d0d-9f88-08bfc4e0dd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:55.836424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8f239-b03a-4d0d-9f88-08bfc4e0dd0e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:55.837086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8f239-b03a-4d0d-9f88-08bfc4e0dd0e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:55.838499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8f239-b03a-4d0d-9f88-08bfc4e0dd0e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-4ab8f239-b03a-4d0d-9f88-08bfc4e0dd0e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:55.852030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:55.853125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:55.854369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:55.857242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 488/982] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:55.859523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f6a90-5a41-4d84-8d5a-e478b64f0dab None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:55.924076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:55.938484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf0f28-4639-455b-a5c5-9366228a1294 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:55.938810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf0f28-4639-455b-a5c5-9366228a1294 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:55.939020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf0f28-4639-455b-a5c5-9366228a1294 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:55.940579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 489/983] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:55.945955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d23d4f-de6e-40fc-bd89-7066f96fa9a0 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:55.946245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d23d4f-de6e-40fc-bd89-7066f96fa9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:55.946522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d23d4f-de6e-40fc-bd89-7066f96fa9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:55.963933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-d0d23d4f-de6e-40fc-bd89-7066f96fa9a0 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:55.965267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d0d23d4f-de6e-40fc-bd89-7066f96fa9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:55.971718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f6a90-5a41-4d84-8d5a-e478b64f0dab None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:56.008350 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe399d29-3762-4bb0-9cac-f6f4a5123e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:56.009021 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe399d29-3762-4bb0-9cac-f6f4a5123e56 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:56.009334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe399d29-3762-4bb0-9cac-f6f4a5123e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:56.009608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe399d29-3762-4bb0-9cac-f6f4a5123e56 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:56.010327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe399d29-3762-4bb0-9cac-f6f4a5123e56 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:56.010628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe399d29-3762-4bb0-9cac-f6f4a5123e56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:56.011990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe399d29-3762-4bb0-9cac-f6f4a5123e56 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:56.017191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 495/984] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jul 14 11:24:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:56.025310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe399d29-3762-4bb0-9cac-f6f4a5123e56 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:56.044506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 490/985] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:56.067604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d23d4f-de6e-40fc-bd89-7066f96fa9a0 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:56.131282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe399d29-3762-4bb0-9cac-f6f4a5123e56 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:56.149544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:56.160435 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:56.168316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf29b9-8209-44f0-8768-ccc1db86fc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:56.168985 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf29b9-8209-44f0-8768-ccc1db86fc02 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:56.169113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d23d4f-de6e-40fc-bd89-7066f96fa9a0 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:56.169334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf29b9-8209-44f0-8768-ccc1db86fc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:56.169559 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf29b9-8209-44f0-8768-ccc1db86fc02 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:56.170264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf29b9-8209-44f0-8768-ccc1db86fc02 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:56.170568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf29b9-8209-44f0-8768-ccc1db86fc02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:56.171935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf29b9-8209-44f0-8768-ccc1db86fc02 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:56.176843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 496/986] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jul 14 11:24:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:56.182482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf29b9-8209-44f0-8768-ccc1db86fc02 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:56.203645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bda69-060e-4035-bcc9-15465a55a02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:56.204321 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bda69-060e-4035-bcc9-15465a55a02f None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:56.204601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bda69-060e-4035-bcc9-15465a55a02f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:56.204878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bda69-060e-4035-bcc9-15465a55a02f None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:56.206158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5bda69-060e-4035-bcc9-15465a55a02f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:56.207538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5bda69-060e-4035-bcc9-15465a55a02f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:56.282627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf29b9-8209-44f0-8768-ccc1db86fc02 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:56.283468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 491/987] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:24:56 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:24:56.328620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027430a5-1cbc-43b2-849b-792847a06df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:56.329332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027430a5-1cbc-43b2-849b-792847a06df3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:56.329612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027430a5-1cbc-43b2-849b-792847a06df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:56.329885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027430a5-1cbc-43b2-849b-792847a06df3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:56.331245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027430a5-1cbc-43b2-849b-792847a06df3 None admin] RBAC: Adding query filter params (interface=admin, service_id=3f6b262c75e94595aee6aec821166499, region_id=RegionOne) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:56.331562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027430a5-1cbc-43b2-849b-792847a06df3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:56.332993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027430a5-1cbc-43b2-849b-792847a06df3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:56.344015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 497/988] 10.222.0.40 () {56 vars in 1249 bytes} [Thu Jul 14 11:24:56 2022] GET /identity/v3/endpoints?service_id=3f6b262c75e94595aee6aec821166499&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:56.345840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bda69-060e-4035-bcc9-15465a55a02f None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:56.462397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bda69-060e-4035-bcc9-15465a55a02f None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:56.498389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca4db7-8566-4b68-9958-ec50b0569788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:56.499089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca4db7-8566-4b68-9958-ec50b0569788 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:56.499375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca4db7-8566-4b68-9958-ec50b0569788 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:56.499646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca4db7-8566-4b68-9958-ec50b0569788 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:56.500329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca4db7-8566-4b68-9958-ec50b0569788 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:56.501827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca4db7-8566-4b68-9958-ec50b0569788 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:56.513238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 492/989] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jul 14 11:24:56 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:24:57.540192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359e1bb-0591-48c7-9c4d-3bcdd7eb868e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:57.540723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359e1bb-0591-48c7-9c4d-3bcdd7eb868e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:57.540999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359e1bb-0591-48c7-9c4d-3bcdd7eb868e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:57.542509 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 498/990] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:57.547643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdf94d-85dc-43d7-a4c8-5768ad6d027d None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:57.548047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdf94d-85dc-43d7-a4c8-5768ad6d027d None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:57.548218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdf94d-85dc-43d7-a4c8-5768ad6d027d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:57.564559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-fafdf94d-85dc-43d7-a4c8-5768ad6d027d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:57.566279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-fafdf94d-85dc-43d7-a4c8-5768ad6d027d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:57.578014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:57.634023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 493/991] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:57.655566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359e1bb-0591-48c7-9c4d-3bcdd7eb868e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:57.700756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:57.761003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359e1bb-0591-48c7-9c4d-3bcdd7eb868e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:57.793620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119dae9-9af5-4673-a93d-6996dcd27f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:57.794331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119dae9-9af5-4673-a93d-6996dcd27f76 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:57.794607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119dae9-9af5-4673-a93d-6996dcd27f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:57.794877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119dae9-9af5-4673-a93d-6996dcd27f76 None admin] PATH_INFO: `/v3/services/image` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:57.798870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119dae9-9af5-4673-a93d-6996dcd27f76 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:57.798870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119dae9-9af5-4673-a93d-6996dcd27f76 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-a119dae9-9af5-4673-a93d-6996dcd27f76 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:57.812230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:57.813288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 11:24:57.814134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:57.817514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 499/992] 10.222.0.40 () {56 vars in 1105 bytes} [Thu Jul 14 11:24:57 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:57.819882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdf94d-85dc-43d7-a4c8-5768ad6d027d None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:57.925596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdf94d-85dc-43d7-a4c8-5768ad6d027d None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:57.964309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dc11e-4c1e-4e4a-b27d-890d57704aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:57.965025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dc11e-4c1e-4e4a-b27d-890d57704aba None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:57.965339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dc11e-4c1e-4e4a-b27d-890d57704aba None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:57.965617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dc11e-4c1e-4e4a-b27d-890d57704aba None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:57.966331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76dc11e-4c1e-4e4a-b27d-890d57704aba None admin] RBAC: Adding query filter params (name=image) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:57.966636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76dc11e-4c1e-4e4a-b27d-890d57704aba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:57.968039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76dc11e-4c1e-4e4a-b27d-890d57704aba None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:57.974081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 494/993] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:57.980030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119dae9-9af5-4673-a93d-6996dcd27f76 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:58.090375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119dae9-9af5-4673-a93d-6996dcd27f76 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:58.097908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:58.125290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d9ed2-a623-49b9-be81-ac8404a76c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:58.125882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d9ed2-a623-49b9-be81-ac8404a76c59 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:58.126160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d9ed2-a623-49b9-be81-ac8404a76c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:58.126428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d9ed2-a623-49b9-be81-ac8404a76c59 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:58.127136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d9ed2-a623-49b9-be81-ac8404a76c59 None admin] RBAC: Adding query filter params (type=image) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:58.127460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d9ed2-a623-49b9-be81-ac8404a76c59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:58.128803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d9ed2-a623-49b9-be81-ac8404a76c59 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:58.141735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 500/994] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:24:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:58.432164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00dd8c-207a-42d2-a308-dfe1451c7da0 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:58.432480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00dd8c-207a-42d2-a308-dfe1451c7da0 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:58.432648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00dd8c-207a-42d2-a308-dfe1451c7da0 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:58.434196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 495/995] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:24:58.439328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7a58e-6b38-4edb-a50b-ba2d9760a52e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:58.439612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7a58e-6b38-4edb-a50b-ba2d9760a52e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:58.439915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7a58e-6b38-4edb-a50b-ba2d9760a52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:58.457417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cec7a58e-6b38-4edb-a50b-ba2d9760a52e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:58.458684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cec7a58e-6b38-4edb-a50b-ba2d9760a52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:58.530064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 501/996] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:24:58.553994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00dd8c-207a-42d2-a308-dfe1451c7da0 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:58.567376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:58.569866 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215006fe-51a2-4346-b1f0-aeeb162059b7 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:58.570307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215006fe-51a2-4346-b1f0-aeeb162059b7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:58.570407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215006fe-51a2-4346-b1f0-aeeb162059b7 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:58.571781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 502/997] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:24:58.577213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87fe08-d34a-4b38-b7b1-c591cbfe4cb8 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:58.577505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87fe08-d34a-4b38-b7b1-c591cbfe4cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:58.580373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87fe08-d34a-4b38-b7b1-c591cbfe4cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:58.594624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-aa87fe08-d34a-4b38-b7b1-c591cbfe4cb8 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:58.596073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-aa87fe08-d34a-4b38-b7b1-c591cbfe4cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:58.666485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00dd8c-207a-42d2-a308-dfe1451c7da0 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:58.669909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 503/998] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:24:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:24:58.693436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87fe08-d34a-4b38-b7b1-c591cbfe4cb8 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:58.710679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a776c-a930-4e89-b078-00cfa3aa2214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:58.711467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a776c-a930-4e89-b078-00cfa3aa2214 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:58.711748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a776c-a930-4e89-b078-00cfa3aa2214 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:58.712016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a776c-a930-4e89-b078-00cfa3aa2214 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:58.712741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a776c-a930-4e89-b078-00cfa3aa2214 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:58.714061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a776c-a930-4e89-b078-00cfa3aa2214 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:58.726077 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-762a776c-a930-4e89-b078-00cfa3aa2214 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:58.726228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:58.727312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:24:58.728161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 496/999] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:24:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:24:58.733115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a776c-a930-4e89-b078-00cfa3aa2214 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:58.800993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87fe08-d34a-4b38-b7b1-c591cbfe4cb8 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:58.819372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:58.841252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880def3-a3b7-4b7d-b02d-c1e67d774b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:58.842056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880def3-a3b7-4b7d-b02d-c1e67d774b8a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:58.842342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880def3-a3b7-4b7d-b02d-c1e67d774b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:58.842617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880def3-a3b7-4b7d-b02d-c1e67d774b8a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:58.843274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880def3-a3b7-4b7d-b02d-c1e67d774b8a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:58.844630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880def3-a3b7-4b7d-b02d-c1e67d774b8a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:58.845812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a776c-a930-4e89-b078-00cfa3aa2214 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-5880def3-a3b7-4b7d-b02d-c1e67d774b8a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:24:58.859002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:24:58.860148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 11:24:58.861014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:24:58.865821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 504/1000] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jul 14 11:24:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:24:58.866722 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880def3-a3b7-4b7d-b02d-c1e67d774b8a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:58.888173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e2a83-a969-4bd7-acd5-55d62e3e8d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:58.888895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e2a83-a969-4bd7-acd5-55d62e3e8d7b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:58.889204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e2a83-a969-4bd7-acd5-55d62e3e8d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:58.889528 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e2a83-a969-4bd7-acd5-55d62e3e8d7b None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:58.890750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e2a83-a969-4bd7-acd5-55d62e3e8d7b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:58.891148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e2a83-a969-4bd7-acd5-55d62e3e8d7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:58.892500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e2a83-a969-4bd7-acd5-55d62e3e8d7b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:58.901936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 497/1001] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:58.902946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e2a83-a969-4bd7-acd5-55d62e3e8d7b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:58.973555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:58.980415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880def3-a3b7-4b7d-b02d-c1e67d774b8a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:59.007691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e2a83-a969-4bd7-acd5-55d62e3e8d7b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:59.019457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53a9c6-58d5-467c-83e7-7e26492541ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:59.020377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a9c6-58d5-467c-83e7-7e26492541ff None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:59.020656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a9c6-58d5-467c-83e7-7e26492541ff None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:59.020928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53a9c6-58d5-467c-83e7-7e26492541ff None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:59.024760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53a9c6-58d5-467c-83e7-7e26492541ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:59.024760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53a9c6-58d5-467c-83e7-7e26492541ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:59.024760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53a9c6-58d5-467c-83e7-7e26492541ff None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:59.031607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 505/1002] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jul 14 11:24:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:24:59.037501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53a9c6-58d5-467c-83e7-7e26492541ff None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:59.048460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e146aee-03bb-4f4f-8ef9-df967f15e851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:59.050000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e146aee-03bb-4f4f-8ef9-df967f15e851 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:59.050477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e146aee-03bb-4f4f-8ef9-df967f15e851 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:59.050943 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e146aee-03bb-4f4f-8ef9-df967f15e851 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:59.051658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e146aee-03bb-4f4f-8ef9-df967f15e851 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:59.051970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e146aee-03bb-4f4f-8ef9-df967f15e851 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:59.053520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e146aee-03bb-4f4f-8ef9-df967f15e851 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:59.064070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 498/1003] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:24:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:59.064963 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e146aee-03bb-4f4f-8ef9-df967f15e851 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:59.068036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:59.140420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53a9c6-58d5-467c-83e7-7e26492541ff None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:59.173250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e146aee-03bb-4f4f-8ef9-df967f15e851 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:59.178702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5eb0e-94c4-4be8-bb6b-28c6e18e6c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:59.179370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5eb0e-94c4-4be8-bb6b-28c6e18e6c41 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:59.179653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5eb0e-94c4-4be8-bb6b-28c6e18e6c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:59.179924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5eb0e-94c4-4be8-bb6b-28c6e18e6c41 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:59.180635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5eb0e-94c4-4be8-bb6b-28c6e18e6c41 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:59.181377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5eb0e-94c4-4be8-bb6b-28c6e18e6c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:59.183163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5eb0e-94c4-4be8-bb6b-28c6e18e6c41 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:59.193010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 506/1004] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jul 14 11:24:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:59.194084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5eb0e-94c4-4be8-bb6b-28c6e18e6c41 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:59.215718 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988545b-bed1-4d40-be63-de5db7e91332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:59.216655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988545b-bed1-4d40-be63-de5db7e91332 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:59.216943 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988545b-bed1-4d40-be63-de5db7e91332 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:59.217249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988545b-bed1-4d40-be63-de5db7e91332 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:59.218522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988545b-bed1-4d40-be63-de5db7e91332 None admin] RBAC: Adding query filter params (interface=public, service_id=7094e5a29d8a4c87b2c560f6744b39e4, region_id=RegionOne) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:59.218839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988545b-bed1-4d40-be63-de5db7e91332 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:59.220384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988545b-bed1-4d40-be63-de5db7e91332 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:59.220835 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:59.230910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 499/1005] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jul 14 11:24:59 2022] GET /identity/v3/endpoints?service_id=7094e5a29d8a4c87b2c560f6744b39e4&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:59.231592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988545b-bed1-4d40-be63-de5db7e91332 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:59.300830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:59.301835 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5eb0e-94c4-4be8-bb6b-28c6e18e6c41 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:59.339054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988545b-bed1-4d40-be63-de5db7e91332 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:59.340643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb144c-1b15-4789-ae19-cb95e06e6985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:59.341444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb144c-1b15-4789-ae19-cb95e06e6985 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:59.341834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb144c-1b15-4789-ae19-cb95e06e6985 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:59.342017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb144c-1b15-4789-ae19-cb95e06e6985 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:59.342780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb144c-1b15-4789-ae19-cb95e06e6985 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:59.344163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb144c-1b15-4789-ae19-cb95e06e6985 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:59.374821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff351ea-2aa9-460b-ab8b-db931662bff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:59.375482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff351ea-2aa9-460b-ab8b-db931662bff2 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:59.375759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff351ea-2aa9-460b-ab8b-db931662bff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:59.376031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff351ea-2aa9-460b-ab8b-db931662bff2 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:59.376713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff351ea-2aa9-460b-ab8b-db931662bff2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:59.378771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff351ea-2aa9-460b-ab8b-db931662bff2 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:59.384931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 500/1006] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jul 14 11:24:59 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:24:59.439015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 507/1007] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jul 14 11:24:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:00.232281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec7efb-9c81-4376-a737-a3eb3b17f1f0 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:00.232787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec7efb-9c81-4376-a737-a3eb3b17f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:00.232787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec7efb-9c81-4376-a737-a3eb3b17f1f0 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:00.234302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 501/1008] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:00.239889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02cd4a-1da5-4a1a-97d6-a4d572f491f9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:00.240370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02cd4a-1da5-4a1a-97d6-a4d572f491f9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:00.240459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02cd4a-1da5-4a1a-97d6-a4d572f491f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:00.257549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8b02cd4a-1da5-4a1a-97d6-a4d572f491f9 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:00.258752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8b02cd4a-1da5-4a1a-97d6-a4d572f491f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:00.335366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 508/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:00 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:00.358024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec7efb-9c81-4376-a737-a3eb3b17f1f0 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:00.467425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec7efb-9c81-4376-a737-a3eb3b17f1f0 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:00.497243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:00.504192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f121411-456d-47ef-b409-d844a81b422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:00.510766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f121411-456d-47ef-b409-d844a81b422e None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:00.510766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f121411-456d-47ef-b409-d844a81b422e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:00.510766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f121411-456d-47ef-b409-d844a81b422e None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:00.510766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f121411-456d-47ef-b409-d844a81b422e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:00.510766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f121411-456d-47ef-b409-d844a81b422e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:00.621578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 502/1010] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:25:00 2022] POST /identity/v3/services => generated 242 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:25:01.505323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178053d9-24fa-4096-9921-85a73215573b None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:01.505895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178053d9-24fa-4096-9921-85a73215573b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:01.505895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178053d9-24fa-4096-9921-85a73215573b None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:01.507298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 509/1011] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:01.513405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dff26-bddc-4e80-80c6-c3ea488d7777 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:01.513946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dff26-bddc-4e80-80c6-c3ea488d7777 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:01.513946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dff26-bddc-4e80-80c6-c3ea488d7777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:01.529978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c02460-1e4f-48d6-9616-9c3038d65265 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:01.530304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c02460-1e4f-48d6-9616-9c3038d65265 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:01.530539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c02460-1e4f-48d6-9616-9c3038d65265 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:01.531944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 510/1012] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:25:01.532489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-f62dff26-bddc-4e80-80c6-c3ea488d7777 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:01.534313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f62dff26-bddc-4e80-80c6-c3ea488d7777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:01.539399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fd8b4-20f6-4b83-8775-11c0bdc7c1e4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:01.539706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fd8b4-20f6-4b83-8775-11c0bdc7c1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:01.539960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fd8b4-20f6-4b83-8775-11c0bdc7c1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:01.559550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-865fd8b4-20f6-4b83-8775-11c0bdc7c1e4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:01.561069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-865fd8b4-20f6-4b83-8775-11c0bdc7c1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:01.563725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:01.635003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 503/1013] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:01 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:01.646518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 511/1014] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:01 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:25:01.659524 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dff26-bddc-4e80-80c6-c3ea488d7777 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:01.669581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fd8b4-20f6-4b83-8775-11c0bdc7c1e4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:01.772298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dff26-bddc-4e80-80c6-c3ea488d7777 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:01.775742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fd8b4-20f6-4b83-8775-11c0bdc7c1e4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:01.789488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:01.813411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08f487-a80e-4d3d-9915-57156578bb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:01.814206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08f487-a80e-4d3d-9915-57156578bb1c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:01.814440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae9ca0-8bd1-4725-9653-ec54f6408c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:01.814579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08f487-a80e-4d3d-9915-57156578bb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:01.814785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08f487-a80e-4d3d-9915-57156578bb1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:01.815267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9ca0-8bd1-4725-9653-ec54f6408c66 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:01.815634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9ca0-8bd1-4725-9653-ec54f6408c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:01.815916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9ca0-8bd1-4725-9653-ec54f6408c66 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:01.816027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08f487-a80e-4d3d-9915-57156578bb1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:01.816576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae9ca0-8bd1-4725-9653-ec54f6408c66 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:01.817631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08f487-a80e-4d3d-9915-57156578bb1c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:01.817981 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae9ca0-8bd1-4725-9653-ec54f6408c66 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:01.819422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 504/1015] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:01.825531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08f487-a80e-4d3d-9915-57156578bb1c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-3fae9ca0-8bd1-4725-9653-ec54f6408c66 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:01.830361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:01.837380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:01.838251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 512/1016] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:25:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:01.839067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae9ca0-8bd1-4725-9653-ec54f6408c66 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:01.841114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:02.009788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae9ca0-8bd1-4725-9653-ec54f6408c66 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:02.009992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08f487-a80e-4d3d-9915-57156578bb1c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:02.047345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42bf9e-0efd-498b-9709-08ba5382721c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:02.047925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42bf9e-0efd-498b-9709-08ba5382721c None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:02.048211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42bf9e-0efd-498b-9709-08ba5382721c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:02.048490 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42bf9e-0efd-498b-9709-08ba5382721c None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:02.049786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42bf9e-0efd-498b-9709-08ba5382721c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:02.049974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85226c-78d5-414b-bb53-ed682d08f4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:02.050630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85226c-78d5-414b-bb53-ed682d08f4fd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:02.050912 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85226c-78d5-414b-bb53-ed682d08f4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:02.051188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85226c-78d5-414b-bb53-ed682d08f4fd None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:02.051284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42bf9e-0efd-498b-9709-08ba5382721c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:02.051902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85226c-78d5-414b-bb53-ed682d08f4fd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:02.052207 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85226c-78d5-414b-bb53-ed682d08f4fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:02.053577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85226c-78d5-414b-bb53-ed682d08f4fd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:02.063840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 513/1017] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:25:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:02.064121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85226c-78d5-414b-bb53-ed682d08f4fd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:02.130334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:02.166377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85226c-78d5-414b-bb53-ed682d08f4fd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:02.168717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 505/1018] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:25:01 2022] POST /identity/v3/projects => generated 311 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:25:02.203578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3cc34-5d2a-4ec7-aa69-06189bd19319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:02.204307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3cc34-5d2a-4ec7-aa69-06189bd19319 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:02.204592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3cc34-5d2a-4ec7-aa69-06189bd19319 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:02.204863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3cc34-5d2a-4ec7-aa69-06189bd19319 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:02.206171 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3cc34-5d2a-4ec7-aa69-06189bd19319 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:02.206481 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3cc34-5d2a-4ec7-aa69-06189bd19319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:02.207908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3cc34-5d2a-4ec7-aa69-06189bd19319 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:02.213681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 514/1019] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jul 14 11:25:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:02.219833 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42bf9e-0efd-498b-9709-08ba5382721c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:02.258126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:02.331624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42bf9e-0efd-498b-9709-08ba5382721c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:02.370311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211e26f-f58c-4de7-a85f-36d12276369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:02.370981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211e26f-f58c-4de7-a85f-36d12276369a None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:02.371267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211e26f-f58c-4de7-a85f-36d12276369a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:02.371557 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211e26f-f58c-4de7-a85f-36d12276369a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:02.372317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211e26f-f58c-4de7-a85f-36d12276369a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:02.373756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211e26f-f58c-4de7-a85f-36d12276369a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:02.453516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 506/1020] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jul 14 11:25:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:02.673037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e930c-5f9e-4fef-bed1-0b4a8658ba09 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:02.673586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e930c-5f9e-4fef-bed1-0b4a8658ba09 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:02.673586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e930c-5f9e-4fef-bed1-0b4a8658ba09 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:02.674948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 515/1021] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:02.680134 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e98bd-6898-41fa-807e-c0bc982aae8c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:02.680569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e98bd-6898-41fa-807e-c0bc982aae8c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:02.680734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e98bd-6898-41fa-807e-c0bc982aae8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:02.698094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-573e98bd-6898-41fa-807e-c0bc982aae8c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:02.699535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-573e98bd-6898-41fa-807e-c0bc982aae8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:02.778127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 507/1022] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:02.800277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e930c-5f9e-4fef-bed1-0b4a8658ba09 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:02.906713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e930c-5f9e-4fef-bed1-0b4a8658ba09 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:02.937246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:02.943587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae784e-e7b2-4b29-91a2-0ffd76daf4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:02.944381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae784e-e7b2-4b29-91a2-0ffd76daf4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:02.944659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae784e-e7b2-4b29-91a2-0ffd76daf4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:02.944928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae784e-e7b2-4b29-91a2-0ffd76daf4d5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:02.945612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae784e-e7b2-4b29-91a2-0ffd76daf4d5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:02.946971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae784e-e7b2-4b29-91a2-0ffd76daf4d5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9bae784e-e7b2-4b29-91a2-0ffd76daf4d5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:02.959094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:02.960134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:02.960983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 516/1023] 10.222.0.40 () {56 vars in 1105 bytes} [Thu Jul 14 11:25:02 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:02.967843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e98bd-6898-41fa-807e-c0bc982aae8c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:02.987797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:03.067190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e98bd-6898-41fa-807e-c0bc982aae8c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:03.102518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8f697-406f-49e7-9ef5-23357341511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:03.103209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8f697-406f-49e7-9ef5-23357341511a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:03.103488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8f697-406f-49e7-9ef5-23357341511a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:03.103760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8f697-406f-49e7-9ef5-23357341511a None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:03.104480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8f697-406f-49e7-9ef5-23357341511a None admin] RBAC: Adding query filter params (name=image) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:03.104864 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8f697-406f-49e7-9ef5-23357341511a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:03.106181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8f697-406f-49e7-9ef5-23357341511a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:03.111318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 508/1024] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:03.116779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae784e-e7b2-4b29-91a2-0ffd76daf4d5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:03.221617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae784e-e7b2-4b29-91a2-0ffd76daf4d5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:03.257000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fa1f7-e32a-41a0-bd3f-d37a8b24251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:03.257687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fa1f7-e32a-41a0-bd3f-d37a8b24251d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:03.257967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fa1f7-e32a-41a0-bd3f-d37a8b24251d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:03.258238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fa1f7-e32a-41a0-bd3f-d37a8b24251d None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:03.258960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fa1f7-e32a-41a0-bd3f-d37a8b24251d None admin] RBAC: Adding query filter params (type=image) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:03.259249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fa1f7-e32a-41a0-bd3f-d37a8b24251d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:03.260612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fa1f7-e32a-41a0-bd3f-d37a8b24251d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:03.271554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 517/1025] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:03.272548 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8f697-406f-49e7-9ef5-23357341511a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:03.357359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:03.376591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8f697-406f-49e7-9ef5-23357341511a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:03.413597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708723c-898a-4821-a4a6-e569c6240855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:03.414270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708723c-898a-4821-a4a6-e569c6240855 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:03.414555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708723c-898a-4821-a4a6-e569c6240855 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:03.414832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708723c-898a-4821-a4a6-e569c6240855 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:03.415633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708723c-898a-4821-a4a6-e569c6240855 None admin] RBAC: Adding query filter params (interface=public, service_id=9476981ca2104bfa884a68ba755d1546, region_id=RegionOne) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:03.415946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708723c-898a-4821-a4a6-e569c6240855 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:03.417348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708723c-898a-4821-a4a6-e569c6240855 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:03.427146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 509/1026] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jul 14 11:25:03 2022] GET /identity/v3/endpoints?service_id=9476981ca2104bfa884a68ba755d1546&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:03.428452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fa1f7-e32a-41a0-bd3f-d37a8b24251d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:03.487306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:03.528947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fa1f7-e32a-41a0-bd3f-d37a8b24251d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:03.563312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b05812-9494-4819-af32-42c0d3d79bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:03.564029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b05812-9494-4819-af32-42c0d3d79bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:03.564257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b05812-9494-4819-af32-42c0d3d79bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:03.564558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b05812-9494-4819-af32-42c0d3d79bd4 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:03.565266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b05812-9494-4819-af32-42c0d3d79bd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:03.566625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b05812-9494-4819-af32-42c0d3d79bd4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:03.581441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 518/1027] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jul 14 11:25:03 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:25:04.288371 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd686f-f4cb-495f-8abe-54b706e91720 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:04.288982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd686f-f4cb-495f-8abe-54b706e91720 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:04.288982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd686f-f4cb-495f-8abe-54b706e91720 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:04.290361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 510/1028] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:04.295639 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295502e8-b9a4-4eeb-8e32-42c5be974c3b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:04.296051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295502e8-b9a4-4eeb-8e32-42c5be974c3b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:04.296218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295502e8-b9a4-4eeb-8e32-42c5be974c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:04.313410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-295502e8-b9a4-4eeb-8e32-42c5be974c3b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:04.314658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-295502e8-b9a4-4eeb-8e32-42c5be974c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:04.403291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 519/1029] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:04.428493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd686f-f4cb-495f-8abe-54b706e91720 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:04.534112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd686f-f4cb-495f-8abe-54b706e91720 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:04.568494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e74173-ff81-4531-844a-006692c45ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:04.569233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e74173-ff81-4531-844a-006692c45ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:04.569545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e74173-ff81-4531-844a-006692c45ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:04.569825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e74173-ff81-4531-844a-006692c45ed5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:04.571016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e74173-ff81-4531-844a-006692c45ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:04.572450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e74173-ff81-4531-844a-006692c45ed5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:04.574370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 511/1030] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:04.580193 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295502e8-b9a4-4eeb-8e32-42c5be974c3b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:04.629143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caead0da-23da-4282-948b-359132fc4e76 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:04.629500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caead0da-23da-4282-948b-359132fc4e76 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:04.630154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caead0da-23da-4282-948b-359132fc4e76 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:04.631673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 512/1031] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:25:04.636225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:04.636628 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c7ae2-18e2-43d6-84c3-9cf58d774bc8 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:04.636916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c7ae2-18e2-43d6-84c3-9cf58d774bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:04.637221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c7ae2-18e2-43d6-84c3-9cf58d774bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:04.653645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-801c7ae2-18e2-43d6-84c3-9cf58d774bc8 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:04.657073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-801c7ae2-18e2-43d6-84c3-9cf58d774bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:04.665306 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:04.686583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295502e8-b9a4-4eeb-8e32-42c5be974c3b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:04.732081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d2c7f-4b41-4be3-8168-e22908ed9927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:04.733027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2c7f-4b41-4be3-8168-e22908ed9927 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:04.733455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2c7f-4b41-4be3-8168-e22908ed9927 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:04.733890 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2c7f-4b41-4be3-8168-e22908ed9927 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:04.734835 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d2c7f-4b41-4be3-8168-e22908ed9927 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:04.736319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d2c7f-4b41-4be3-8168-e22908ed9927 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:04.741455 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 513/1032] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:25:04.768983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c7ae2-18e2-43d6-84c3-9cf58d774bc8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:04.864449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 520/1033] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:25:04 2022] POST /identity/v3/users => generated 285 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:04.873790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c7ae2-18e2-43d6-84c3-9cf58d774bc8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:04.915248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14018ed-2ecb-4251-910f-d5fe49f756e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:04.916079 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14018ed-2ecb-4251-910f-d5fe49f756e0 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:04.916357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14018ed-2ecb-4251-910f-d5fe49f756e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:04.916624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14018ed-2ecb-4251-910f-d5fe49f756e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:04.929538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14018ed-2ecb-4251-910f-d5fe49f756e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-b14018ed-2ecb-4251-910f-d5fe49f756e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:04.937917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:04.939080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:04.940298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 514/1034] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jul 14 11:25:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:04.945615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d2c7f-4b41-4be3-8168-e22908ed9927 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:05.059074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d2c7f-4b41-4be3-8168-e22908ed9927 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:05.068926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:05.094628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb2fc1-e56c-4d3a-b2ec-dc43d4da9c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:05.095313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb2fc1-e56c-4d3a-b2ec-dc43d4da9c78 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:05.095652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb2fc1-e56c-4d3a-b2ec-dc43d4da9c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:05.096494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb2fc1-e56c-4d3a-b2ec-dc43d4da9c78 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:05.097271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb2fc1-e56c-4d3a-b2ec-dc43d4da9c78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:05.097633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb2fc1-e56c-4d3a-b2ec-dc43d4da9c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:05.099020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb2fc1-e56c-4d3a-b2ec-dc43d4da9c78 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:05.110015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 521/1035] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jul 14 11:25:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:05.115385 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14018ed-2ecb-4251-910f-d5fe49f756e0 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:05.119854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:05.218635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14018ed-2ecb-4251-910f-d5fe49f756e0 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:05.251204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa791f5-6457-48da-81fd-f4649c37a9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:05.252515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa791f5-6457-48da-81fd-f4649c37a9be None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:05.252793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa791f5-6457-48da-81fd-f4649c37a9be None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:05.253062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa791f5-6457-48da-81fd-f4649c37a9be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:05.264101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa791f5-6457-48da-81fd-f4649c37a9be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:05.265577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa791f5-6457-48da-81fd-f4649c37a9be None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-9fa791f5-6457-48da-81fd-f4649c37a9be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:25:05.283216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:05.284185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:05.285676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:25:05.285676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:25:05.285676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:05.285676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:05.285676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 515/1036] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:05.290147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb2fc1-e56c-4d3a-b2ec-dc43d4da9c78 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:05.392057 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb2fc1-e56c-4d3a-b2ec-dc43d4da9c78 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:05.430609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055537b-36a3-4bb5-af31-36581e2a89a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:05.431281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055537b-36a3-4bb5-af31-36581e2a89a5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:05.431562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055537b-36a3-4bb5-af31-36581e2a89a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:05.431837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055537b-36a3-4bb5-af31-36581e2a89a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:05.432563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055537b-36a3-4bb5-af31-36581e2a89a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:05.432870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055537b-36a3-4bb5-af31-36581e2a89a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:05.434265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055537b-36a3-4bb5-af31-36581e2a89a5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:05.450880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 522/1037] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:05.452385 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa791f5-6457-48da-81fd-f4649c37a9be None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:05.511352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:05.554986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa791f5-6457-48da-81fd-f4649c37a9be None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:05.590112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bb044-95c5-402b-a13f-3523f2cb79df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:05.590908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb044-95c5-402b-a13f-3523f2cb79df None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:05.591191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb044-95c5-402b-a13f-3523f2cb79df None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:05.591465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb044-95c5-402b-a13f-3523f2cb79df None admin] PATH_INFO: `/v3/users/nova` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:05.599890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bb044-95c5-402b-a13f-3523f2cb79df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:05.601368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bb044-95c5-402b-a13f-3523f2cb79df None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-ee4bb044-95c5-402b-a13f-3523f2cb79df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:25:05.615468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:05.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:05.622374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 516/1038] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jul 14 11:25:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:05.624457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055537b-36a3-4bb5-af31-36581e2a89a5 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:05.657755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:05.658691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed2590-1a60-4f89-89a4-79537b3325a2 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:05.658994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed2590-1a60-4f89-89a4-79537b3325a2 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:05.659272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed2590-1a60-4f89-89a4-79537b3325a2 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:05.660655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 517/1039] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:05.665603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9743ae3-6b04-4b56-a603-09118460b652 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:05.669306 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9743ae3-6b04-4b56-a603-09118460b652 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:05.669306 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9743ae3-6b04-4b56-a603-09118460b652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:05.683214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c9743ae3-6b04-4b56-a603-09118460b652 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:05.684423 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c9743ae3-6b04-4b56-a603-09118460b652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:05.733513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055537b-36a3-4bb5-af31-36581e2a89a5 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:05.755070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 518/1040] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:25:05.771956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a5764-afc6-45d0-886f-3abe5ac870cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:05.772626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a5764-afc6-45d0-886f-3abe5ac870cc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:05.772906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a5764-afc6-45d0-886f-3abe5ac870cc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:05.773238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a5764-afc6-45d0-886f-3abe5ac870cc None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:05.774079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a5764-afc6-45d0-886f-3abe5ac870cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:05.774385 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a5764-afc6-45d0-886f-3abe5ac870cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:05.775817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a5764-afc6-45d0-886f-3abe5ac870cc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:05.780694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9743ae3-6b04-4b56-a603-09118460b652 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:05.805482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 523/1041] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jul 14 11:25:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:05.811760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a5764-afc6-45d0-886f-3abe5ac870cc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:05.841548 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:05.905422 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:05.906958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9743ae3-6b04-4b56-a603-09118460b652 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:05.918186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a5764-afc6-45d0-886f-3abe5ac870cc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:05.942959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5cb12-f6ad-4905-9cbc-c8803e39ae82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:05.943745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5cb12-f6ad-4905-9cbc-c8803e39ae82 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:05.944023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5cb12-f6ad-4905-9cbc-c8803e39ae82 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:05.944353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5cb12-f6ad-4905-9cbc-c8803e39ae82 None admin] PATH_INFO: `/v3/services/image` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:05.944928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5cb12-f6ad-4905-9cbc-c8803e39ae82 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:05.946323 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5cb12-f6ad-4905-9cbc-c8803e39ae82 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:05.955347 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe4712-85e0-450f-9c78-41d446a888c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:05.956001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe4712-85e0-450f-9c78-41d446a888c3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:05.956832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe4712-85e0-450f-9c78-41d446a888c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:05.957107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe4712-85e0-450f-9c78-41d446a888c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-a5f5cb12-f6ad-4905-9cbc-c8803e39ae82 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:05.959479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:05.960478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:05.961399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 519/1042] 10.222.0.40 () {56 vars in 1105 bytes} [Thu Jul 14 11:25:05 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:05.966574 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5cb12-f6ad-4905-9cbc-c8803e39ae82 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:06.038427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe4712-85e0-450f-9c78-41d446a888c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:06.039992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe4712-85e0-450f-9c78-41d446a888c3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:06.073801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5cb12-f6ad-4905-9cbc-c8803e39ae82 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:06.085312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:06.114787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee24c7-b012-40c2-8d08-e8ef99e7f7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:06.115562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee24c7-b012-40c2-8d08-e8ef99e7f7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:06.115869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee24c7-b012-40c2-8d08-e8ef99e7f7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:06.116490 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee24c7-b012-40c2-8d08-e8ef99e7f7a2 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:06.117456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee24c7-b012-40c2-8d08-e8ef99e7f7a2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:06.118068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee24c7-b012-40c2-8d08-e8ef99e7f7a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:06.119906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee24c7-b012-40c2-8d08-e8ef99e7f7a2 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:06.125503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 520/1043] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:06.131647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee24c7-b012-40c2-8d08-e8ef99e7f7a2 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:06.149533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:06.164779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 524/1044] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 11:25:06.170488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe4712-85e0-450f-9c78-41d446a888c3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:06.242605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee24c7-b012-40c2-8d08-e8ef99e7f7a2 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:06.276673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe4712-85e0-450f-9c78-41d446a888c3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:06.277626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beab426-5666-4bcb-bebe-b789b5190ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:06.278291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beab426-5666-4bcb-bebe-b789b5190ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:06.278577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beab426-5666-4bcb-bebe-b789b5190ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:06.278856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beab426-5666-4bcb-bebe-b789b5190ea3 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:06.279708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beab426-5666-4bcb-bebe-b789b5190ea3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:06.279891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beab426-5666-4bcb-bebe-b789b5190ea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:06.281256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beab426-5666-4bcb-bebe-b789b5190ea3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:06.286678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 521/1045] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:06.292156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beab426-5666-4bcb-bebe-b789b5190ea3 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:06.318291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a17ba7-7e7b-4b74-a5c5-b3eaada35d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:06.319084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17ba7-7e7b-4b74-a5c5-b3eaada35d91 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:06.319367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17ba7-7e7b-4b74-a5c5-b3eaada35d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:06.319643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17ba7-7e7b-4b74-a5c5-b3eaada35d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:06.330007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a17ba7-7e7b-4b74-a5c5-b3eaada35d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:06.331442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a17ba7-7e7b-4b74-a5c5-b3eaada35d91 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:06.336230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-55a17ba7-7e7b-4b74-a5c5-b3eaada35d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:25:06.349574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:06.359397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:06.360330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:25:06.360330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:25:06.360330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:06.360330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:06.360330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 525/1046] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:06.360330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a17ba7-7e7b-4b74-a5c5-b3eaada35d91 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:06.404325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:06.409062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beab426-5666-4bcb-bebe-b789b5190ea3 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:06.452773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cb8f3-9bb7-4f2a-9237-48293d0ada52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:06.453789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cb8f3-9bb7-4f2a-9237-48293d0ada52 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:06.453789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cb8f3-9bb7-4f2a-9237-48293d0ada52 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:06.453933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cb8f3-9bb7-4f2a-9237-48293d0ada52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:06.454658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cb8f3-9bb7-4f2a-9237-48293d0ada52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:06.456079 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cb8f3-9bb7-4f2a-9237-48293d0ada52 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:06.481898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a17ba7-7e7b-4b74-a5c5-b3eaada35d91 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:06.516080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509c218-3085-48c6-9b6c-61ca688d8ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:06.516742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509c218-3085-48c6-9b6c-61ca688d8ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:06.517032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509c218-3085-48c6-9b6c-61ca688d8ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:06.517385 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509c218-3085-48c6-9b6c-61ca688d8ab3 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:06.518626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509c218-3085-48c6-9b6c-61ca688d8ab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:06.518936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509c218-3085-48c6-9b6c-61ca688d8ab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:06.520312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509c218-3085-48c6-9b6c-61ca688d8ab3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:06.531991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 526/1047] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:06.538167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509c218-3085-48c6-9b6c-61ca688d8ab3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:06.634740 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:06.637756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509c218-3085-48c6-9b6c-61ca688d8ab3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:06.661197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 522/1048] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jul 14 11:25:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:06.680537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ec539-d783-415f-be43-cfec5c87eed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:06.681380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ec539-d783-415f-be43-cfec5c87eed3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:06.681659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ec539-d783-415f-be43-cfec5c87eed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:06.681930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ec539-d783-415f-be43-cfec5c87eed3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:06.690432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ec539-d783-415f-be43-cfec5c87eed3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:06.691919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ec539-d783-415f-be43-cfec5c87eed3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-d02ec539-d783-415f-be43-cfec5c87eed3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:25:06.708115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:06.709166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:06.710440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:25:06.710440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:25:06.710440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:25:06.710440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:25:06.710440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:25:06.710440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:25:06.710440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:25:06.710440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:06.710440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 527/1049] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:25:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:06.715690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cb8f3-9bb7-4f2a-9237-48293d0ada52 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:06.821495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:06.831239 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cb8f3-9bb7-4f2a-9237-48293d0ada52 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:06.868389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1d907-b728-49c1-8828-9a68c315e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:06.869061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1d907-b728-49c1-8828-9a68c315e06b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:06.869417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1d907-b728-49c1-8828-9a68c315e06b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:06.874178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1d907-b728-49c1-8828-9a68c315e06b None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:06.874178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1d907-b728-49c1-8828-9a68c315e06b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:06.874178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1d907-b728-49c1-8828-9a68c315e06b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:06.874178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1d907-b728-49c1-8828-9a68c315e06b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:06.884948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 523/1050] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:25:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:06.891177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ec539-d783-415f-be43-cfec5c87eed3 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:06.995904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a36d3b-1e01-4434-a09a-040f557f3d87 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:06.995904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a36d3b-1e01-4434-a09a-040f557f3d87 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:06.996328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a36d3b-1e01-4434-a09a-040f557f3d87 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:06.997809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 524/1051] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:07.003030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ec539-d783-415f-be43-cfec5c87eed3 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:07.003546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd335e0-6c1c-4d5f-aab0-082c01427010 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:07.003836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd335e0-6c1c-4d5f-aab0-082c01427010 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:07.004115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd335e0-6c1c-4d5f-aab0-082c01427010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:07.021252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ecd335e0-6c1c-4d5f-aab0-082c01427010 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:07.022440 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ecd335e0-6c1c-4d5f-aab0-082c01427010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:07.040452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf612e-a8a2-4727-a57d-4b6f053346a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:07.041524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf612e-a8a2-4727-a57d-4b6f053346a0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:07.041812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf612e-a8a2-4727-a57d-4b6f053346a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:07.042088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf612e-a8a2-4727-a57d-4b6f053346a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:07.042925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf612e-a8a2-4727-a57d-4b6f053346a0 None admin] RBAC: Adding query filter params (role.id=b7e771a1c5f54085a2a9174024f86f19, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=147d764ef17042dbb7ea179c29711588) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:07.043238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf612e-a8a2-4727-a57d-4b6f053346a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:07.045210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf612e-a8a2-4727-a57d-4b6f053346a0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:07.051818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 528/1052] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:25:06 2022] GET /identity/v3/role_assignments?user.id=147d764ef17042dbb7ea179c29711588&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=b7e771a1c5f54085a2a9174024f86f19 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:07.108590 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 525/1053] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:25:07.133241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf612e-a8a2-4727-a57d-4b6f053346a0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:07.159420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:07.244857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf612e-a8a2-4727-a57d-4b6f053346a0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:07.282306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c611a-866b-4c88-9de5-bd913deaace9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:07.283119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c611a-866b-4c88-9de5-bd913deaace9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:07.283396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c611a-866b-4c88-9de5-bd913deaace9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:07.283665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c611a-866b-4c88-9de5-bd913deaace9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:07.293872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c611a-866b-4c88-9de5-bd913deaace9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-9a9c611a-866b-4c88-9de5-bd913deaace9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:07.301966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:07.302920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:07.303804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 529/1054] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:25:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:07.316907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd335e0-6c1c-4d5f-aab0-082c01427010 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:07.397194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:07.414364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd335e0-6c1c-4d5f-aab0-082c01427010 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:07.450060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b7d92-7fac-4a48-a102-381ff0ca9aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:07.450863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7d92-7fac-4a48-a102-381ff0ca9aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:07.451218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7d92-7fac-4a48-a102-381ff0ca9aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:07.451570 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7d92-7fac-4a48-a102-381ff0ca9aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:07.452403 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b7d92-7fac-4a48-a102-381ff0ca9aa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:07.452786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b7d92-7fac-4a48-a102-381ff0ca9aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:07.454269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b7d92-7fac-4a48-a102-381ff0ca9aa7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:07.467725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 526/1055] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:25:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:07.469222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c611a-866b-4c88-9de5-bd913deaace9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:07.559501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:07.578089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c611a-866b-4c88-9de5-bd913deaace9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:07.610976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b00c1-f034-4939-a433-32b0fe659dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:07.613029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b00c1-f034-4939-a433-32b0fe659dde None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:07.613388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b00c1-f034-4939-a433-32b0fe659dde None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:07.613856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b00c1-f034-4939-a433-32b0fe659dde None admin] PATH_INFO: `/v3/users/7dcab6a1ddbe44389936bc537977adb0` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:07.627684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b00c1-f034-4939-a433-32b0fe659dde None admin] RBAC: Authorizing `identity:get_user(user_id=7dcab6a1ddbe44389936bc537977adb0)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:07.629179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b00c1-f034-4939-a433-32b0fe659dde None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:07.635683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 530/1056] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:07 2022] GET /identity/v3/users/7dcab6a1ddbe44389936bc537977adb0 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:07.637067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b7d92-7fac-4a48-a102-381ff0ca9aa7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:07.744831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b7d92-7fac-4a48-a102-381ff0ca9aa7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:07.778826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a922e-2d14-443d-a1ad-2338eee1611e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:07.779473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a922e-2d14-443d-a1ad-2338eee1611e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:07.779765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a922e-2d14-443d-a1ad-2338eee1611e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:07.780041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a922e-2d14-443d-a1ad-2338eee1611e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:07.799149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:07.856095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a922e-2d14-443d-a1ad-2338eee1611e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:07.857550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a922e-2d14-443d-a1ad-2338eee1611e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:07.959199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 527/1057] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:25:07.964835 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b00c1-f034-4939-a433-32b0fe659dde None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:08.064308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b00c1-f034-4939-a433-32b0fe659dde None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:08.099762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26f706-612e-4f88-b0cc-cc7337f4f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:08.100623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26f706-612e-4f88-b0cc-cc7337f4f4ea None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:08.100904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26f706-612e-4f88-b0cc-cc7337f4f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:08.101202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26f706-612e-4f88-b0cc-cc7337f4f4ea None admin] PATH_INFO: `/v3/projects/f09928d8e8054beca93d734e72641b8e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:08.102381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26f706-612e-4f88-b0cc-cc7337f4f4ea None admin] RBAC: Authorizing `identity:get_project(project_id=f09928d8e8054beca93d734e72641b8e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:08.103809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26f706-612e-4f88-b0cc-cc7337f4f4ea None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:08.105756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 531/1058] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:07 2022] GET /identity/v3/projects/f09928d8e8054beca93d734e72641b8e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:08.111720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a922e-2d14-443d-a1ad-2338eee1611e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:08.161998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:08.221345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a922e-2d14-443d-a1ad-2338eee1611e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:08.260088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508afd0-9dc9-42a0-9c7a-21513ede3a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:08.260742 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508afd0-9dc9-42a0-9c7a-21513ede3a16 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:08.261027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508afd0-9dc9-42a0-9c7a-21513ede3a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:08.261334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508afd0-9dc9-42a0-9c7a-21513ede3a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:08.262171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508afd0-9dc9-42a0-9c7a-21513ede3a16 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=f09928d8e8054beca93d734e72641b8e, user.id=7dcab6a1ddbe44389936bc537977adb0) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:08.262487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508afd0-9dc9-42a0-9c7a-21513ede3a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:08.263927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508afd0-9dc9-42a0-9c7a-21513ede3a16 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:08.269736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 528/1059] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:25:08 2022] GET /identity/v3/role_assignments?user.id=7dcab6a1ddbe44389936bc537977adb0&scope.project.id=f09928d8e8054beca93d734e72641b8e&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:08.746967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b9e76-97c3-406e-bad6-fb4b71b0c56d None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:08.747493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b9e76-97c3-406e-bad6-fb4b71b0c56d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:08.747493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b9e76-97c3-406e-bad6-fb4b71b0c56d None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:08.749061 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 532/1060] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:08.754694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906455-39e1-4398-bcc2-b63cb168dcba None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:08.755142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906455-39e1-4398-bcc2-b63cb168dcba None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:08.755266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906455-39e1-4398-bcc2-b63cb168dcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:08.772236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e6906455-39e1-4398-bcc2-b63cb168dcba None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:08.774154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e6906455-39e1-4398-bcc2-b63cb168dcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:08.846768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 529/1061] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:08.870041 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b9e76-97c3-406e-bad6-fb4b71b0c56d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:08.879992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:08.976386 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b9e76-97c3-406e-bad6-fb4b71b0c56d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:09.012095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aa27d-92ec-4d13-9f75-4161a4fc2055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:09.013050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aa27d-92ec-4d13-9f75-4161a4fc2055 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:09.013556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aa27d-92ec-4d13-9f75-4161a4fc2055 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:09.014009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aa27d-92ec-4d13-9f75-4161a4fc2055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:09.061257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c746d-4d18-4c58-8a43-0ddd5352a603 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:09.061555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c746d-4d18-4c58-8a43-0ddd5352a603 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:09.061820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c746d-4d18-4c58-8a43-0ddd5352a603 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:09.063194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 530/1062] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:09.068122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4932-183c-44f7-921a-a5241a2156bf None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:09.068414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4932-183c-44f7-921a-a5241a2156bf None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:09.068694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4932-183c-44f7-921a-a5241a2156bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:09.084968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-be1e4932-183c-44f7-921a-a5241a2156bf None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:09.085805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492aa27d-92ec-4d13-9f75-4161a4fc2055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:09.086192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-be1e4932-183c-44f7-921a-a5241a2156bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:09.087714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492aa27d-92ec-4d13-9f75-4161a4fc2055 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:09.096793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:09.120941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:09.152448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 531/1063] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:25:09.176115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e4932-183c-44f7-921a-a5241a2156bf None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:09.196871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 533/1064] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:25:09.202877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aa27d-92ec-4d13-9f75-4161a4fc2055 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:09.284276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e4932-183c-44f7-921a-a5241a2156bf None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:09.310312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aa27d-92ec-4d13-9f75-4161a4fc2055 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:09.317999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:09.318345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e25458-006a-4577-a8c7-93060e203ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:09.319154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e25458-006a-4577-a8c7-93060e203ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:09.319440 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e25458-006a-4577-a8c7-93060e203ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:09.319716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e25458-006a-4577-a8c7-93060e203ef4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:09.331075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e25458-006a-4577-a8c7-93060e203ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-58e25458-006a-4577-a8c7-93060e203ef4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:09.339650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:09.340717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:09.341624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 532/1065] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jul 14 11:25:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:09.348004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e25458-006a-4577-a8c7-93060e203ef4 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:09.352770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888dc85-b528-4ca9-a6f6-b685b28f6c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:09.353705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888dc85-b528-4ca9-a6f6-b685b28f6c3a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:09.353988 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888dc85-b528-4ca9-a6f6-b685b28f6c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:09.354257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888dc85-b528-4ca9-a6f6-b685b28f6c3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:09.368841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c888dc85-b528-4ca9-a6f6-b685b28f6c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:09.370737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c888dc85-b528-4ca9-a6f6-b685b28f6c3a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:09.382096 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-c888dc85-b528-4ca9-a6f6-b685b28f6c3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:25:09.389727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:09.390869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:09.391779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:25:09.391779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:25:09.391779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:09.391779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:09.391779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 534/1066] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:09.397497 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888dc85-b528-4ca9-a6f6-b685b28f6c3a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:09.461618 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e25458-006a-4577-a8c7-93060e203ef4 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:09.498535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5ff05-235f-4cd5-811a-b2c4f3f595aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:09.499210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5ff05-235f-4cd5-811a-b2c4f3f595aa None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:09.499494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5ff05-235f-4cd5-811a-b2c4f3f595aa None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:09.499770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5ff05-235f-4cd5-811a-b2c4f3f595aa None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:09.500526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5ff05-235f-4cd5-811a-b2c4f3f595aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:09.500830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5ff05-235f-4cd5-811a-b2c4f3f595aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:09.502274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5ff05-235f-4cd5-811a-b2c4f3f595aa None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:09.510714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888dc85-b528-4ca9-a6f6-b685b28f6c3a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:09.515679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 533/1067] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jul 14 11:25:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:09.516617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5ff05-235f-4cd5-811a-b2c4f3f595aa None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:09.545863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ce2f4-207a-44d9-b7d7-5d25cadfd6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:09.546444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ce2f4-207a-44d9-b7d7-5d25cadfd6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:09.546726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ce2f4-207a-44d9-b7d7-5d25cadfd6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:09.546998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ce2f4-207a-44d9-b7d7-5d25cadfd6d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:09.547712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ce2f4-207a-44d9-b7d7-5d25cadfd6d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:09.548020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ce2f4-207a-44d9-b7d7-5d25cadfd6d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:09.549426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ce2f4-207a-44d9-b7d7-5d25cadfd6d0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:09.561140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 535/1068] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:09.620355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:09.630686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5ff05-235f-4cd5-811a-b2c4f3f595aa None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:09.669798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce4f67-66a6-41b8-b3f1-53113ea45e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:09.670619 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce4f67-66a6-41b8-b3f1-53113ea45e00 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:09.671513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce4f67-66a6-41b8-b3f1-53113ea45e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:09.671513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce4f67-66a6-41b8-b3f1-53113ea45e00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:09.685735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce4f67-66a6-41b8-b3f1-53113ea45e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:09.687246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce4f67-66a6-41b8-b3f1-53113ea45e00 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-0fce4f67-66a6-41b8-b3f1-53113ea45e00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:25:09.707977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:09.708980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:09.714438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:25:09.714438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:25:09.714438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:09.714438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:09.714438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 534/1069] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:09.730402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ce2f4-207a-44d9-b7d7-5d25cadfd6d0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:09.765196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:09.877999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ce2f4-207a-44d9-b7d7-5d25cadfd6d0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:09.915132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc7ba-8771-44b4-8555-e4ddf158ba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:09.915810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc7ba-8771-44b4-8555-e4ddf158ba4d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:09.916092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc7ba-8771-44b4-8555-e4ddf158ba4d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:09.916370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cc7ba-8771-44b4-8555-e4ddf158ba4d None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:09.917191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cc7ba-8771-44b4-8555-e4ddf158ba4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:09.917561 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cc7ba-8771-44b4-8555-e4ddf158ba4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:09.918986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cc7ba-8771-44b4-8555-e4ddf158ba4d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:09.932135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 536/1070] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:09.937934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce4f67-66a6-41b8-b3f1-53113ea45e00 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.046516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce4f67-66a6-41b8-b3f1-53113ea45e00 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:10.087362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ba1a6-3ce1-4bfb-9579-4d79ba2b0b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:10.088080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba1a6-3ce1-4bfb-9579-4d79ba2b0b41 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.088363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba1a6-3ce1-4bfb-9579-4d79ba2b0b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.088638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba1a6-3ce1-4bfb-9579-4d79ba2b0b41 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.096404 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ba1a6-3ce1-4bfb-9579-4d79ba2b0b41 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:10.097887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ba1a6-3ce1-4bfb-9579-4d79ba2b0b41 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-f22ba1a6-3ce1-4bfb-9579-4d79ba2b0b41 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:25:10.112806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:10.114007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:10.115011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 535/1071] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jul 14 11:25:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:10.119760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc7ba-8771-44b4-8555-e4ddf158ba4d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.201785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:10.224019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cc7ba-8771-44b4-8555-e4ddf158ba4d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:10.259859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67db3c-9e39-43dd-8096-badb021e63fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:10.260525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67db3c-9e39-43dd-8096-badb021e63fd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.260840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67db3c-9e39-43dd-8096-badb021e63fd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.261111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67db3c-9e39-43dd-8096-badb021e63fd None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.261982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67db3c-9e39-43dd-8096-badb021e63fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:10.262292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67db3c-9e39-43dd-8096-badb021e63fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:10.263716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67db3c-9e39-43dd-8096-badb021e63fd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:10.281117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 537/1072] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jul 14 11:25:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:10.287127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ba1a6-3ce1-4bfb-9579-4d79ba2b0b41 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.290882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:10.306186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62280b48-75cb-430e-b3d3-bb1df79cf972 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.306642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62280b48-75cb-430e-b3d3-bb1df79cf972 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.307075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62280b48-75cb-430e-b3d3-bb1df79cf972 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.308601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 538/1073] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:25:10.313651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9600c-5ebe-479b-96df-162bd534bfc7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.313943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9600c-5ebe-479b-96df-162bd534bfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.314251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9600c-5ebe-479b-96df-162bd534bfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.331569 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5dd9600c-5ebe-479b-96df-162bd534bfc7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:10.332925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5dd9600c-5ebe-479b-96df-162bd534bfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:10.387045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ba1a6-3ce1-4bfb-9579-4d79ba2b0b41 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:10.399665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 539/1074] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:25:10.422809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9600c-5ebe-479b-96df-162bd534bfc7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.423219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caada1d0-d47a-4bd3-82f4-1c0bb3f85e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:10.424038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caada1d0-d47a-4bd3-82f4-1c0bb3f85e38 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.424323 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caada1d0-d47a-4bd3-82f4-1c0bb3f85e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.424680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caada1d0-d47a-4bd3-82f4-1c0bb3f85e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.435393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caada1d0-d47a-4bd3-82f4-1c0bb3f85e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:10.436845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caada1d0-d47a-4bd3-82f4-1c0bb3f85e38 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-caada1d0-d47a-4bd3-82f4-1c0bb3f85e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:25:10.454659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:10.456177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:10.457313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:25:10.457313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:25:10.457313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:10.457313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:10.457313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 536/1075] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:10.462018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caada1d0-d47a-4bd3-82f4-1c0bb3f85e38 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.488805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:10.516451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:10.522664 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9600c-5ebe-479b-96df-162bd534bfc7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:10.556536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be71fb-f5f7-4a28-954f-e7ee5724d5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:10.557300 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be71fb-f5f7-4a28-954f-e7ee5724d5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.557584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be71fb-f5f7-4a28-954f-e7ee5724d5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.557856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be71fb-f5f7-4a28-954f-e7ee5724d5c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.559333 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caada1d0-d47a-4bd3-82f4-1c0bb3f85e38 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:10.568742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be71fb-f5f7-4a28-954f-e7ee5724d5c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-a5be71fb-f5f7-4a28-954f-e7ee5724d5c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:10.576613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:10.577654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:10.578524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 540/1076] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:25:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:10.584617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be71fb-f5f7-4a28-954f-e7ee5724d5c0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.594632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745910-5c5d-405f-bccf-74c09d43fa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:10.595326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745910-5c5d-405f-bccf-74c09d43fa83 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.595613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745910-5c5d-405f-bccf-74c09d43fa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.595890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745910-5c5d-405f-bccf-74c09d43fa83 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.596624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63745910-5c5d-405f-bccf-74c09d43fa83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:10.596932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63745910-5c5d-405f-bccf-74c09d43fa83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:10.598348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63745910-5c5d-405f-bccf-74c09d43fa83 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:10.610111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 537/1077] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:10.616085 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745910-5c5d-405f-bccf-74c09d43fa83 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.684223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be71fb-f5f7-4a28-954f-e7ee5724d5c0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:10.709936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:10.712395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745910-5c5d-405f-bccf-74c09d43fa83 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:10.718133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e9674-8c3f-4f73-952a-042aae218772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:10.718796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e9674-8c3f-4f73-952a-042aae218772 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.719076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e9674-8c3f-4f73-952a-042aae218772 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.719347 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e9674-8c3f-4f73-952a-042aae218772 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.720141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e9674-8c3f-4f73-952a-042aae218772 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:10.720464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e9674-8c3f-4f73-952a-042aae218772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:10.721864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e9674-8c3f-4f73-952a-042aae218772 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:10.730106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:10.730394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 541/1078] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:25:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:10.736788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e9674-8c3f-4f73-952a-042aae218772 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.746234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c4a54-578a-440f-ade2-71898321469e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:10.747025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c4a54-578a-440f-ade2-71898321469e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.747307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c4a54-578a-440f-ade2-71898321469e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.747581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c4a54-578a-440f-ade2-71898321469e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.754844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c4a54-578a-440f-ade2-71898321469e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:10.756290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c4a54-578a-440f-ade2-71898321469e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-7e2c4a54-578a-440f-ade2-71898321469e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:25:10.770521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:10.771476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:10.772391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:25:10.772391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:25:10.772391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:25:10.772391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:25:10.772391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:25:10.772391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:25:10.772391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:25:10.772391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:10.772391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 538/1079] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:25:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:10.777605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c4a54-578a-440f-ade2-71898321469e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.838874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e9674-8c3f-4f73-952a-042aae218772 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:10.873475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaae84-490f-4a53-bfa0-78c4e064734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:10.874429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaae84-490f-4a53-bfa0-78c4e064734d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.874619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaae84-490f-4a53-bfa0-78c4e064734d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.874891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaae84-490f-4a53-bfa0-78c4e064734d None admin] PATH_INFO: `/v3/users/7dcab6a1ddbe44389936bc537977adb0` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.876097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aaae84-490f-4a53-bfa0-78c4e064734d None admin] RBAC: Authorizing `identity:get_user(user_id=7dcab6a1ddbe44389936bc537977adb0)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:10.878119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aaae84-490f-4a53-bfa0-78c4e064734d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:10.879756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 542/1080] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:10 2022] GET /identity/v3/users/7dcab6a1ddbe44389936bc537977adb0 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:10.881029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c4a54-578a-440f-ade2-71898321469e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:10.885424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaae84-490f-4a53-bfa0-78c4e064734d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.914187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae512d3-4e07-4096-bea4-d2621c068d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:10.914836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae512d3-4e07-4096-bea4-d2621c068d8e None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:10.915114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae512d3-4e07-4096-bea4-d2621c068d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:10.915383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae512d3-4e07-4096-bea4-d2621c068d8e None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:10.916162 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae512d3-4e07-4096-bea4-d2621c068d8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:10.916467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae512d3-4e07-4096-bea4-d2621c068d8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:10.917937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae512d3-4e07-4096-bea4-d2621c068d8e None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:10.929749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 539/1081] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:25:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:10.935546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae512d3-4e07-4096-bea4-d2621c068d8e None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:10.974470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:10.984489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:11.017661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaae84-490f-4a53-bfa0-78c4e064734d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:11.047921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae512d3-4e07-4096-bea4-d2621c068d8e None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:11.053243 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6629b9-a20c-4180-a063-1c76df0239d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:11.054097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6629b9-a20c-4180-a063-1c76df0239d7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:11.054375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6629b9-a20c-4180-a063-1c76df0239d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:11.054648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6629b9-a20c-4180-a063-1c76df0239d7 None admin] PATH_INFO: `/v3/projects/f09928d8e8054beca93d734e72641b8e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:11.055817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6629b9-a20c-4180-a063-1c76df0239d7 None admin] RBAC: Authorizing `identity:get_project(project_id=f09928d8e8054beca93d734e72641b8e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:11.057293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6629b9-a20c-4180-a063-1c76df0239d7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:11.059000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 543/1082] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:10 2022] GET /identity/v3/projects/f09928d8e8054beca93d734e72641b8e => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:11.065368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6629b9-a20c-4180-a063-1c76df0239d7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:11.084416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4db5e-5f51-469b-9d0e-7e2b8f927781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:11.085335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4db5e-5f51-469b-9d0e-7e2b8f927781 None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:11.085622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4db5e-5f51-469b-9d0e-7e2b8f927781 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:11.085896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4db5e-5f51-469b-9d0e-7e2b8f927781 None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/147d764ef17042dbb7ea179c29711588/roles/b7e771a1c5f54085a2a9174024f86f19` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:11.088172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4db5e-5f51-469b-9d0e-7e2b8f927781 None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=147d764ef17042dbb7ea179c29711588, role_id=b7e771a1c5f54085a2a9174024f86f19)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:11.089722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4db5e-5f51-469b-9d0e-7e2b8f927781 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:11.161627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6629b9-a20c-4180-a063-1c76df0239d7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:11.184619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:11.194829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb1d3a-493c-497f-8b62-261e234d00e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:11.195643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb1d3a-493c-497f-8b62-261e234d00e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:11.195921 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb1d3a-493c-497f-8b62-261e234d00e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:11.196184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb1d3a-493c-497f-8b62-261e234d00e8 None admin] PATH_INFO: `/v3/projects/f09928d8e8054beca93d734e72641b8e/users/7dcab6a1ddbe44389936bc537977adb0/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:11.198153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb1d3a-493c-497f-8b62-261e234d00e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f09928d8e8054beca93d734e72641b8e, user_id=7dcab6a1ddbe44389936bc537977adb0, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:11.199822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb1d3a-493c-497f-8b62-261e234d00e8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:11.235349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 540/1083] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:25:10 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/147d764ef17042dbb7ea179c29711588/roles/b7e771a1c5f54085a2a9174024f86f19 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:25:11.332404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 544/1084] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:25:11 2022] PUT /identity/v3/projects/f09928d8e8054beca93d734e72641b8e/users/7dcab6a1ddbe44389936bc537977adb0/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:25:13.233016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e592b9-64aa-4576-b7cd-5374ec633417 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.233548 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e592b9-64aa-4576-b7cd-5374ec633417 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.233548 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e592b9-64aa-4576-b7cd-5374ec633417 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.235023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 541/1085] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:13.240306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd64101-0322-494b-9e02-36a499e15dc3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.240777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd64101-0322-494b-9e02-36a499e15dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.240979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd64101-0322-494b-9e02-36a499e15dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.257960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8bd64101-0322-494b-9e02-36a499e15dc3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:13.259029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8bd64101-0322-494b-9e02-36a499e15dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:13.326538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 545/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:25:13.350724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e592b9-64aa-4576-b7cd-5374ec633417 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:13.383910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153fe54-9c97-48b1-bc0d-24cc5c295549 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.384198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153fe54-9c97-48b1-bc0d-24cc5c295549 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.384473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153fe54-9c97-48b1-bc0d-24cc5c295549 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.385911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 546/1087] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:25:13.391061 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377af94-685e-4500-ab9b-2a15dd75b07a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.391334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377af94-685e-4500-ab9b-2a15dd75b07a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.391713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377af94-685e-4500-ab9b-2a15dd75b07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.411419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a377af94-685e-4500-ab9b-2a15dd75b07a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:13.412622 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a377af94-685e-4500-ab9b-2a15dd75b07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:13.447805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:13.459551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e592b9-64aa-4576-b7cd-5374ec633417 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:13.480237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 547/1088] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:25:13.493177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e0bdd-4886-42c8-9566-4cb811d165fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:13.493926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e0bdd-4886-42c8-9566-4cb811d165fc None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.494432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e0bdd-4886-42c8-9566-4cb811d165fc None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.495059 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e0bdd-4886-42c8-9566-4cb811d165fc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.504486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377af94-685e-4500-ab9b-2a15dd75b07a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:13.505782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e0bdd-4886-42c8-9566-4cb811d165fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-de7e0bdd-4886-42c8-9566-4cb811d165fc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:13.513933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:13.515052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 11:25:13.516094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:13.517954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 542/1089] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jul 14 11:25:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:13.521658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e0bdd-4886-42c8-9566-4cb811d165fc None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:13.549480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:13.604393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377af94-685e-4500-ab9b-2a15dd75b07a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:13.618785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e0bdd-4886-42c8-9566-4cb811d165fc None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:13.637974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4099eb-44f7-4bc0-83d1-11f73e91f88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:13.638753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4099eb-44f7-4bc0-83d1-11f73e91f88e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.639028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4099eb-44f7-4bc0-83d1-11f73e91f88e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.639296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4099eb-44f7-4bc0-83d1-11f73e91f88e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.649690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4099eb-44f7-4bc0-83d1-11f73e91f88e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:13.652040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a7891-551b-4f40-af18-258e529601dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:13.652697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a7891-551b-4f40-af18-258e529601dd None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.652980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a7891-551b-4f40-af18-258e529601dd None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.653306 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a7891-551b-4f40-af18-258e529601dd None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.654055 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a7891-551b-4f40-af18-258e529601dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:13.654365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a7891-551b-4f40-af18-258e529601dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:13.655730 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a7891-551b-4f40-af18-258e529601dd None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-dc4099eb-44f7-4bc0-83d1-11f73e91f88e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:13.657298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:13.659002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:13.660608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 548/1090] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:25:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:13.664867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4099eb-44f7-4bc0-83d1-11f73e91f88e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:13.669336 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 543/1091] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jul 14 11:25:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:13.670703 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a7891-551b-4f40-af18-258e529601dd None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:13.689559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:13.764476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4099eb-44f7-4bc0-83d1-11f73e91f88e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:13.766814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:13.774536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a7891-551b-4f40-af18-258e529601dd None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:13.798597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3379159-d3ff-43ee-9c09-8518efe97f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:13.799801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3379159-d3ff-43ee-9c09-8518efe97f18 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.800036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3379159-d3ff-43ee-9c09-8518efe97f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.800365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3379159-d3ff-43ee-9c09-8518efe97f18 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.801110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3379159-d3ff-43ee-9c09-8518efe97f18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:13.801907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3379159-d3ff-43ee-9c09-8518efe97f18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:13.802798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3379159-d3ff-43ee-9c09-8518efe97f18 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:13.809947 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4607b-8fb1-4494-acf1-6aa1bf1ede2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:13.810728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4607b-8fb1-4494-acf1-6aa1bf1ede2d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.811011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4607b-8fb1-4494-acf1-6aa1bf1ede2d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.811284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 549/1092] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:25:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:13.811416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4607b-8fb1-4494-acf1-6aa1bf1ede2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.817491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3379159-d3ff-43ee-9c09-8518efe97f18 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:13.822595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4607b-8fb1-4494-acf1-6aa1bf1ede2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:13.824319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4607b-8fb1-4494-acf1-6aa1bf1ede2d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-f0b4607b-8fb1-4494-acf1-6aa1bf1ede2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:25:13.842265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:13.843200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:13.844108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:25:13.844108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:25:13.844108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:13.844108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:13.844108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 544/1093] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:13.848991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4607b-8fb1-4494-acf1-6aa1bf1ede2d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:13.917719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:13.919929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3379159-d3ff-43ee-9c09-8518efe97f18 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:13.948842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4607b-8fb1-4494-acf1-6aa1bf1ede2d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:13.954230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787db5b-1820-464a-a9ef-b4cfcc946e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:13.955227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787db5b-1820-464a-a9ef-b4cfcc946e42 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.955505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787db5b-1820-464a-a9ef-b4cfcc946e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.955775 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787db5b-1820-464a-a9ef-b4cfcc946e42 None admin] PATH_INFO: `/v3/users/7dcab6a1ddbe44389936bc537977adb0` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.957005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787db5b-1820-464a-a9ef-b4cfcc946e42 None admin] RBAC: Authorizing `identity:get_user(user_id=7dcab6a1ddbe44389936bc537977adb0)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:13.958469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787db5b-1820-464a-a9ef-b4cfcc946e42 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:13.960148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 550/1094] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:13 2022] GET /identity/v3/users/7dcab6a1ddbe44389936bc537977adb0 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:13.965811 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787db5b-1820-464a-a9ef-b4cfcc946e42 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:13.984574 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b1625-d74d-4b76-a77b-c8188eacd730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:13.985318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b1625-d74d-4b76-a77b-c8188eacd730 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:13.985604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b1625-d74d-4b76-a77b-c8188eacd730 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:13.986415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b1625-d74d-4b76-a77b-c8188eacd730 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:13.987149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b1625-d74d-4b76-a77b-c8188eacd730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:13.987459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b1625-d74d-4b76-a77b-c8188eacd730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:13.988869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b1625-d74d-4b76-a77b-c8188eacd730 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:14.000238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:14.001352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 545/1095] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:14.006954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b1625-d74d-4b76-a77b-c8188eacd730 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:14.067714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787db5b-1820-464a-a9ef-b4cfcc946e42 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:14.100863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787590c6-cff5-4566-a8a2-df0b269f07bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:14.101536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787590c6-cff5-4566-a8a2-df0b269f07bf None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:14.101810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787590c6-cff5-4566-a8a2-df0b269f07bf None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:14.102080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787590c6-cff5-4566-a8a2-df0b269f07bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:14.106065 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b1625-d74d-4b76-a77b-c8188eacd730 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:14.134883 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:14.141018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc80290-25c8-4f82-ae53-029d72d33662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:14.141837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc80290-25c8-4f82-ae53-029d72d33662 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:14.142140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc80290-25c8-4f82-ae53-029d72d33662 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:14.142410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc80290-25c8-4f82-ae53-029d72d33662 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:14.149397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc80290-25c8-4f82-ae53-029d72d33662 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:14.150834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc80290-25c8-4f82-ae53-029d72d33662 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-8cc80290-25c8-4f82-ae53-029d72d33662 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:25:14.165922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:14.166929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:14.167983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 546/1096] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jul 14 11:25:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:25:14.168590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787590c6-cff5-4566-a8a2-df0b269f07bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:14.169869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787590c6-cff5-4566-a8a2-df0b269f07bf None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:14.174744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc80290-25c8-4f82-ae53-029d72d33662 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:14.202905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:14.283051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 551/1097] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 11:25:14.288979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787590c6-cff5-4566-a8a2-df0b269f07bf None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:14.291130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc80290-25c8-4f82-ae53-029d72d33662 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:14.330588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd32a50-ccd2-48a3-b2e3-d0476c4e5e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:14.331257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd32a50-ccd2-48a3-b2e3-d0476c4e5e64 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:14.331604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd32a50-ccd2-48a3-b2e3-d0476c4e5e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:14.331945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd32a50-ccd2-48a3-b2e3-d0476c4e5e64 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:14.332918 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd32a50-ccd2-48a3-b2e3-d0476c4e5e64 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:14.333362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd32a50-ccd2-48a3-b2e3-d0476c4e5e64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:14.335104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd32a50-ccd2-48a3-b2e3-d0476c4e5e64 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:14.352893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 547/1098] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jul 14 11:25:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:14.359227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd32a50-ccd2-48a3-b2e3-d0476c4e5e64 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:14.400613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787590c6-cff5-4566-a8a2-df0b269f07bf None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:14.411964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:14.437383 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bd6bf-76ac-45f6-8032-9ad47e79377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:14.438013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bd6bf-76ac-45f6-8032-9ad47e79377a None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:14.438518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bd6bf-76ac-45f6-8032-9ad47e79377a None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:14.438518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bd6bf-76ac-45f6-8032-9ad47e79377a None admin] PATH_INFO: `/v3/projects/f09928d8e8054beca93d734e72641b8e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:14.440296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bd6bf-76ac-45f6-8032-9ad47e79377a None admin] RBAC: Authorizing `identity:get_project(project_id=f09928d8e8054beca93d734e72641b8e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:14.441805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bd6bf-76ac-45f6-8032-9ad47e79377a None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:14.443530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 552/1099] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:14 2022] GET /identity/v3/projects/f09928d8e8054beca93d734e72641b8e => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:14.450038 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bd6bf-76ac-45f6-8032-9ad47e79377a None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:14.452860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:14.468601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd32a50-ccd2-48a3-b2e3-d0476c4e5e64 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:14.512447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb853e05-ffaa-44e7-acce-4760598ab0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:14.513113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb853e05-ffaa-44e7-acce-4760598ab0db None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:14.513422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb853e05-ffaa-44e7-acce-4760598ab0db None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:14.513695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb853e05-ffaa-44e7-acce-4760598ab0db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:14.556324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bd6bf-76ac-45f6-8032-9ad47e79377a None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:14.591328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb853e05-ffaa-44e7-acce-4760598ab0db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:14.592706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb853e05-ffaa-44e7-acce-4760598ab0db None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:14.595152 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361d51c-c95c-4b43-9fe2-47aadca341fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:14.595955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361d51c-c95c-4b43-9fe2-47aadca341fe None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:14.596259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361d51c-c95c-4b43-9fe2-47aadca341fe None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:14.596719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361d51c-c95c-4b43-9fe2-47aadca341fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:14.597769 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361d51c-c95c-4b43-9fe2-47aadca341fe None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=f09928d8e8054beca93d734e72641b8e, user.id=7dcab6a1ddbe44389936bc537977adb0) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:14.598243 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361d51c-c95c-4b43-9fe2-47aadca341fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:14.600230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361d51c-c95c-4b43-9fe2-47aadca341fe None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:14.606706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 553/1100] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:25:14 2022] GET /identity/v3/role_assignments?user.id=7dcab6a1ddbe44389936bc537977adb0&scope.project.id=f09928d8e8054beca93d734e72641b8e&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:14.649359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:14.711512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 548/1101] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 11:25:14.717511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361d51c-c95c-4b43-9fe2-47aadca341fe None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:14.802278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:14.837365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361d51c-c95c-4b43-9fe2-47aadca341fe None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:14.872410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdef882-15da-4cd0-97a8-3d0dd64c1587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:14.873120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdef882-15da-4cd0-97a8-3d0dd64c1587 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:14.873427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdef882-15da-4cd0-97a8-3d0dd64c1587 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:14.873693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdef882-15da-4cd0-97a8-3d0dd64c1587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:14.886746 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdef882-15da-4cd0-97a8-3d0dd64c1587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:14.888176 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdef882-15da-4cd0-97a8-3d0dd64c1587 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-acdef882-15da-4cd0-97a8-3d0dd64c1587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:25:14.904922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:14.905983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:14.906900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:25:14.906900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:25:14.906900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:25:14.906900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:14.906900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 554/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:14.912560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb853e05-ffaa-44e7-acce-4760598ab0db None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:15.011469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb853e05-ffaa-44e7-acce-4760598ab0db None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:15.045045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e26c-5838-430f-8c01-f7297de1a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:15.045728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e26c-5838-430f-8c01-f7297de1a92c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:15.046013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e26c-5838-430f-8c01-f7297de1a92c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:15.046287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e26c-5838-430f-8c01-f7297de1a92c None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:15.047009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417e26c-5838-430f-8c01-f7297de1a92c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:15.047316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417e26c-5838-430f-8c01-f7297de1a92c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:15.048749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417e26c-5838-430f-8c01-f7297de1a92c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:15.060262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 549/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:15.067975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdef882-15da-4cd0-97a8-3d0dd64c1587 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:15.166636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdef882-15da-4cd0-97a8-3d0dd64c1587 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:15.182918 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:15.199714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d299b3-a40b-4634-aac4-d04051ff9df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:15.200505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d299b3-a40b-4634-aac4-d04051ff9df0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:15.200779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d299b3-a40b-4634-aac4-d04051ff9df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:15.201049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d299b3-a40b-4634-aac4-d04051ff9df0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:15.207657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d299b3-a40b-4634-aac4-d04051ff9df0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:15.209075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d299b3-a40b-4634-aac4-d04051ff9df0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-73d299b3-a40b-4634-aac4-d04051ff9df0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:25:15.223781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:15.224860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:15.230748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:25:15.230748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:25:15.230748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:25:15.230748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:25:15.230748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:25:15.230748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:25:15.230748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:25:15.230748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:15.230748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 555/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:15.231255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e26c-5838-430f-8c01-f7297de1a92c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:15.242997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:15.331238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e26c-5838-430f-8c01-f7297de1a92c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:15.364021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653bf28-9afa-44d3-9125-e39a88f017df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:15.364683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653bf28-9afa-44d3-9125-e39a88f017df None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:15.365069 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653bf28-9afa-44d3-9125-e39a88f017df None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:15.365298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653bf28-9afa-44d3-9125-e39a88f017df None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:15.366074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653bf28-9afa-44d3-9125-e39a88f017df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:15.366386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653bf28-9afa-44d3-9125-e39a88f017df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:15.367833 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653bf28-9afa-44d3-9125-e39a88f017df None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:15.380276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 550/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:25:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:15.386390 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d299b3-a40b-4634-aac4-d04051ff9df0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:15.493589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d299b3-a40b-4634-aac4-d04051ff9df0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:15.526861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19caca6-17ea-4d26-b8b2-b79e76f324b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:15.527537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19caca6-17ea-4d26-b8b2-b79e76f324b3 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:15.527821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19caca6-17ea-4d26-b8b2-b79e76f324b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:15.528087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19caca6-17ea-4d26-b8b2-b79e76f324b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:15.528917 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19caca6-17ea-4d26-b8b2-b79e76f324b3 None admin] RBAC: Adding query filter params (role.id=b7e771a1c5f54085a2a9174024f86f19, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=147d764ef17042dbb7ea179c29711588) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:15.529273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19caca6-17ea-4d26-b8b2-b79e76f324b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:15.530717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19caca6-17ea-4d26-b8b2-b79e76f324b3 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:15.537265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 556/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:25:15 2022] GET /identity/v3/role_assignments?user.id=147d764ef17042dbb7ea179c29711588&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=b7e771a1c5f54085a2a9174024f86f19 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:16.753668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43da25-ea67-477a-80be-57b9dc94eac9 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:16.754197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43da25-ea67-477a-80be-57b9dc94eac9 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:16.754290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43da25-ea67-477a-80be-57b9dc94eac9 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:16.755655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 551/1107] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:16.760736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bf8dd-8e88-4cad-b140-2d1e9e1f34af None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:16.761196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bf8dd-8e88-4cad-b140-2d1e9e1f34af None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:16.761318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bf8dd-8e88-4cad-b140-2d1e9e1f34af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:16.777752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a88bf8dd-8e88-4cad-b140-2d1e9e1f34af None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:16.778926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a88bf8dd-8e88-4cad-b140-2d1e9e1f34af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:16.814475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:16.844867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 557/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:16.870850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43da25-ea67-477a-80be-57b9dc94eac9 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:16.946493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:16.980973 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43da25-ea67-477a-80be-57b9dc94eac9 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:17.013298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe0dd1-e328-4e05-8979-cb90baf160e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:17.014080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe0dd1-e328-4e05-8979-cb90baf160e5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:17.014365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe0dd1-e328-4e05-8979-cb90baf160e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:17.014640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe0dd1-e328-4e05-8979-cb90baf160e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:17.015810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe0dd1-e328-4e05-8979-cb90baf160e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:17.017816 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe0dd1-e328-4e05-8979-cb90baf160e5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:17.019426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 552/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:17.025325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bf8dd-8e88-4cad-b140-2d1e9e1f34af None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:17.120589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bf8dd-8e88-4cad-b140-2d1e9e1f34af None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:17.153953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e4d9f-f9c7-4f19-a458-86fe12a53f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:17.154609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e4d9f-f9c7-4f19-a458-86fe12a53f54 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:17.154893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e4d9f-f9c7-4f19-a458-86fe12a53f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:17.155170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e4d9f-f9c7-4f19-a458-86fe12a53f54 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:17.155889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e4d9f-f9c7-4f19-a458-86fe12a53f54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:17.157342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e4d9f-f9c7-4f19-a458-86fe12a53f54 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:17.265294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 558/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:25:17 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:19.346966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363f924-c6c4-4d1f-ad5d-821c7615e96f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:19.347509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363f924-c6c4-4d1f-ad5d-821c7615e96f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:19.347791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363f924-c6c4-4d1f-ad5d-821c7615e96f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:19.349942 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 553/1111] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:19.355844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0f548-e880-4f44-9be3-3fe7a7d82b4d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:19.356333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0f548-e880-4f44-9be3-3fe7a7d82b4d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:19.356522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0f548-e880-4f44-9be3-3fe7a7d82b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:19.373902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b6f0f548-e880-4f44-9be3-3fe7a7d82b4d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:19.375091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b6f0f548-e880-4f44-9be3-3fe7a7d82b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:19.398880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:19.443582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 559/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:25:19.467596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363f924-c6c4-4d1f-ad5d-821c7615e96f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:19.569387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363f924-c6c4-4d1f-ad5d-821c7615e96f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:19.604539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329203b4-fc08-4976-8062-5427b5b0a888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:19.605334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329203b4-fc08-4976-8062-5427b5b0a888 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:19.605616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329203b4-fc08-4976-8062-5427b5b0a888 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:19.605890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329203b4-fc08-4976-8062-5427b5b0a888 None admin] PATH_INFO: `/v3/roles/41fd0da9b16d41c08992073473d87488` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:19.607466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329203b4-fc08-4976-8062-5427b5b0a888 None admin] RBAC: Authorizing `identity:get_role(role_id=41fd0da9b16d41c08992073473d87488)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:19.608834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329203b4-fc08-4976-8062-5427b5b0a888 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:19.610270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 554/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:19 2022] GET /identity/v3/roles/41fd0da9b16d41c08992073473d87488 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:19.616643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0f548-e880-4f44-9be3-3fe7a7d82b4d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:19.729261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0f548-e880-4f44-9be3-3fe7a7d82b4d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:19.762678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89f892-4d02-47c8-964d-390a2f349ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:19.763528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f892-4d02-47c8-964d-390a2f349ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:19.763813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f892-4d02-47c8-964d-390a2f349ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:19.764136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f892-4d02-47c8-964d-390a2f349ff0 None admin] PATH_INFO: `/v3/users/250d4c8e98574ef29b5672702970f305` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:19.780756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89f892-4d02-47c8-964d-390a2f349ff0 None admin] RBAC: Authorizing `identity:get_user(user_id=250d4c8e98574ef29b5672702970f305)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:19.782252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89f892-4d02-47c8-964d-390a2f349ff0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:19.785015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 560/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:19 2022] GET /identity/v3/users/250d4c8e98574ef29b5672702970f305 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:19.790814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329203b4-fc08-4976-8062-5427b5b0a888 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:19.794565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:19.890184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329203b4-fc08-4976-8062-5427b5b0a888 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:19.918686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3011c-e9f3-46c6-96cb-6cf15e454dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:19.919315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3011c-e9f3-46c6-96cb-6cf15e454dfe None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:19.919595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3011c-e9f3-46c6-96cb-6cf15e454dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:19.919866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3011c-e9f3-46c6-96cb-6cf15e454dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:19.985241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3011c-e9f3-46c6-96cb-6cf15e454dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:19.987680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3011c-e9f3-46c6-96cb-6cf15e454dfe None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:19.990397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:20.121473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 555/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:25:20.127914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89f892-4d02-47c8-964d-390a2f349ff0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:20.146122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:20.250892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89f892-4d02-47c8-964d-390a2f349ff0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:20.284201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcbc3d-0271-42eb-a96b-1b7f3d3dd23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:20.285040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcbc3d-0271-42eb-a96b-1b7f3d3dd23f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:20.285352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcbc3d-0271-42eb-a96b-1b7f3d3dd23f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:20.285631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcbc3d-0271-42eb-a96b-1b7f3d3dd23f None admin] PATH_INFO: `/v3/projects/f09928d8e8054beca93d734e72641b8e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:20.286799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcbc3d-0271-42eb-a96b-1b7f3d3dd23f None admin] RBAC: Authorizing `identity:get_project(project_id=f09928d8e8054beca93d734e72641b8e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:20.288225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcbc3d-0271-42eb-a96b-1b7f3d3dd23f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:20.289948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 561/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:20 2022] GET /identity/v3/projects/f09928d8e8054beca93d734e72641b8e => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:20.296192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3011c-e9f3-46c6-96cb-6cf15e454dfe None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:20.416403 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3011c-e9f3-46c6-96cb-6cf15e454dfe None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:20.418638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:20.449725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c674a-7196-4661-aa58-0d675b66db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:20.450379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c674a-7196-4661-aa58-0d675b66db91 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:20.450858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c674a-7196-4661-aa58-0d675b66db91 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:20.451354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c674a-7196-4661-aa58-0d675b66db91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:20.452507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c674a-7196-4661-aa58-0d675b66db91 None admin] RBAC: Adding query filter params (role.id=41fd0da9b16d41c08992073473d87488, scope.project.id=f09928d8e8054beca93d734e72641b8e, user.id=250d4c8e98574ef29b5672702970f305) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:20.452833 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c674a-7196-4661-aa58-0d675b66db91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:20.454337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c674a-7196-4661-aa58-0d675b66db91 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:20.459656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 556/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:25:20 2022] GET /identity/v3/role_assignments?user.id=250d4c8e98574ef29b5672702970f305&scope.project.id=f09928d8e8054beca93d734e72641b8e&role.id=41fd0da9b16d41c08992073473d87488 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:22.512089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257d24-18a9-48c8-bc08-a9c257883fd6 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:22.512716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257d24-18a9-48c8-bc08-a9c257883fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:22.512716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257d24-18a9-48c8-bc08-a9c257883fd6 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:22.514028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 562/1118] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:22.519320 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee874886-5adf-4c09-9a81-52ef5a07d244 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:22.519762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee874886-5adf-4c09-9a81-52ef5a07d244 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:22.519865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee874886-5adf-4c09-9a81-52ef5a07d244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:22.536594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ee874886-5adf-4c09-9a81-52ef5a07d244 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:22.537819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ee874886-5adf-4c09-9a81-52ef5a07d244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:22.603335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 557/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:22.626591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257d24-18a9-48c8-bc08-a9c257883fd6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:22.707668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:22.724101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257d24-18a9-48c8-bc08-a9c257883fd6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:22.757459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66439ec0-d09b-40e0-9e5d-b3887701c0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:22.758312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66439ec0-d09b-40e0-9e5d-b3887701c0ce None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:22.758595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66439ec0-d09b-40e0-9e5d-b3887701c0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:22.758868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66439ec0-d09b-40e0-9e5d-b3887701c0ce None admin] PATH_INFO: `/v3/roles/41fd0da9b16d41c08992073473d87488` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:22.760350 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66439ec0-d09b-40e0-9e5d-b3887701c0ce None admin] RBAC: Authorizing `identity:get_role(role_id=41fd0da9b16d41c08992073473d87488)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:22.761750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66439ec0-d09b-40e0-9e5d-b3887701c0ce None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:22.763162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 563/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:22 2022] GET /identity/v3/roles/41fd0da9b16d41c08992073473d87488 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:22.769469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee874886-5adf-4c09-9a81-52ef5a07d244 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:22.860349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:22.863845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee874886-5adf-4c09-9a81-52ef5a07d244 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:22.896139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c0f47-31cb-48c4-9582-7a13a46a2b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:22.897010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c0f47-31cb-48c4-9582-7a13a46a2b3c None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:22.897409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c0f47-31cb-48c4-9582-7a13a46a2b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:22.897893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c0f47-31cb-48c4-9582-7a13a46a2b3c None admin] PATH_INFO: `/v3/users/250d4c8e98574ef29b5672702970f305` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:22.899489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c0f47-31cb-48c4-9582-7a13a46a2b3c None admin] RBAC: Authorizing `identity:get_user(user_id=250d4c8e98574ef29b5672702970f305)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:22.900928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c0f47-31cb-48c4-9582-7a13a46a2b3c None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:22.902601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 558/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:22 2022] GET /identity/v3/users/250d4c8e98574ef29b5672702970f305 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:22.908955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66439ec0-d09b-40e0-9e5d-b3887701c0ce None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:23.006255 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66439ec0-d09b-40e0-9e5d-b3887701c0ce None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:23.039317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b00a0-81f0-4fab-b33c-d21ba56a7082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:23.040096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b00a0-81f0-4fab-b33c-d21ba56a7082 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:23.040379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b00a0-81f0-4fab-b33c-d21ba56a7082 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:23.040654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b00a0-81f0-4fab-b33c-d21ba56a7082 None admin] PATH_INFO: `/v3/projects/f09928d8e8054beca93d734e72641b8e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:23.041844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b00a0-81f0-4fab-b33c-d21ba56a7082 None admin] RBAC: Authorizing `identity:get_project(project_id=f09928d8e8054beca93d734e72641b8e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:23.043263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b00a0-81f0-4fab-b33c-d21ba56a7082 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:23.044954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 564/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:22 2022] GET /identity/v3/projects/f09928d8e8054beca93d734e72641b8e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:23.051042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c0f47-31cb-48c4-9582-7a13a46a2b3c None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:23.148133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c0f47-31cb-48c4-9582-7a13a46a2b3c None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:23.180763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf586593-bdb0-4e7c-8173-a5a0f697464f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:23.181660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf586593-bdb0-4e7c-8173-a5a0f697464f None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:23.181944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf586593-bdb0-4e7c-8173-a5a0f697464f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:23.182214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf586593-bdb0-4e7c-8173-a5a0f697464f None admin] PATH_INFO: `/v3/projects/f09928d8e8054beca93d734e72641b8e/users/250d4c8e98574ef29b5672702970f305/roles/41fd0da9b16d41c08992073473d87488` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:23.184446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf586593-bdb0-4e7c-8173-a5a0f697464f None admin] RBAC: Authorizing `identity:create_grant(project_id=f09928d8e8054beca93d734e72641b8e, user_id=250d4c8e98574ef29b5672702970f305, role_id=41fd0da9b16d41c08992073473d87488)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:23.185910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf586593-bdb0-4e7c-8173-a5a0f697464f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:23.268851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 559/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:25:23 2022] PUT /identity/v3/projects/f09928d8e8054beca93d734e72641b8e/users/250d4c8e98574ef29b5672702970f305/roles/41fd0da9b16d41c08992073473d87488 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:25:25.256238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed32e5-6b67-4163-ac4c-ef0744d465fa None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:25.256762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed32e5-6b67-4163-ac4c-ef0744d465fa None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:25.256762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed32e5-6b67-4163-ac4c-ef0744d465fa None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:25.258159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 565/1124] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:25.263197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b830842-e35b-4e56-81fe-b3f5e7392c8b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:25.263608 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b830842-e35b-4e56-81fe-b3f5e7392c8b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:25.263751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b830842-e35b-4e56-81fe-b3f5e7392c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:25.280040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-5b830842-e35b-4e56-81fe-b3f5e7392c8b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:25.281252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5b830842-e35b-4e56-81fe-b3f5e7392c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:25.301236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:25.346104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 560/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:25.371152 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed32e5-6b67-4163-ac4c-ef0744d465fa None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:25.392946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:25.471584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed32e5-6b67-4163-ac4c-ef0744d465fa None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:25.504788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbf53e-eecb-4dc2-bb64-1fe68e23f577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:25.505649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf53e-eecb-4dc2-bb64-1fe68e23f577 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:25.505929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf53e-eecb-4dc2-bb64-1fe68e23f577 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:25.506200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf53e-eecb-4dc2-bb64-1fe68e23f577 None admin] PATH_INFO: `/v3/roles/41fd0da9b16d41c08992073473d87488` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:25.507747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbf53e-eecb-4dc2-bb64-1fe68e23f577 None admin] RBAC: Authorizing `identity:get_role(role_id=41fd0da9b16d41c08992073473d87488)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:25.509046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbf53e-eecb-4dc2-bb64-1fe68e23f577 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:25.510437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 566/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:25 2022] GET /identity/v3/roles/41fd0da9b16d41c08992073473d87488 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:25.516344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b830842-e35b-4e56-81fe-b3f5e7392c8b None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:25.611908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b830842-e35b-4e56-81fe-b3f5e7392c8b None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:25.643692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619f826-60e1-44fb-9b06-9698acaad7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:25.644539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619f826-60e1-44fb-9b06-9698acaad7fc None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:25.644826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619f826-60e1-44fb-9b06-9698acaad7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:25.645100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619f826-60e1-44fb-9b06-9698acaad7fc None admin] PATH_INFO: `/v3/users/250d4c8e98574ef29b5672702970f305` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:25.646304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619f826-60e1-44fb-9b06-9698acaad7fc None admin] RBAC: Authorizing `identity:get_user(user_id=250d4c8e98574ef29b5672702970f305)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:25.647726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619f826-60e1-44fb-9b06-9698acaad7fc None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:25.649355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 561/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:25 2022] GET /identity/v3/users/250d4c8e98574ef29b5672702970f305 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:25.655317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbf53e-eecb-4dc2-bb64-1fe68e23f577 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:25.738809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:25.749962 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbf53e-eecb-4dc2-bb64-1fe68e23f577 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:25.782860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e6234-04de-4fdf-8e09-954b0894905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:25.783428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e6234-04de-4fdf-8e09-954b0894905c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:25.783705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e6234-04de-4fdf-8e09-954b0894905c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:25.783979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e6234-04de-4fdf-8e09-954b0894905c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:25.852363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e6234-04de-4fdf-8e09-954b0894905c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:25.853852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e6234-04de-4fdf-8e09-954b0894905c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:25.936492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:25.948752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 567/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:25:25.955296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619f826-60e1-44fb-9b06-9698acaad7fc None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:25.980453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:26.050979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619f826-60e1-44fb-9b06-9698acaad7fc None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:26.083255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bf3d9-410e-4b30-b99f-de760319ee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:26.084091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bf3d9-410e-4b30-b99f-de760319ee8f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:26.084366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bf3d9-410e-4b30-b99f-de760319ee8f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:26.084632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bf3d9-410e-4b30-b99f-de760319ee8f None admin] PATH_INFO: `/v3/projects/f09928d8e8054beca93d734e72641b8e` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:26.085799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bf3d9-410e-4b30-b99f-de760319ee8f None admin] RBAC: Authorizing `identity:get_project(project_id=f09928d8e8054beca93d734e72641b8e)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:26.087221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bf3d9-410e-4b30-b99f-de760319ee8f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:26.088878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 562/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:25 2022] GET /identity/v3/projects/f09928d8e8054beca93d734e72641b8e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:26.095138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e6234-04de-4fdf-8e09-954b0894905c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:26.190725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e6234-04de-4fdf-8e09-954b0894905c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:26.223948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4ac53-4253-44c2-ac65-f86b9b848e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:26.224596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ac53-4253-44c2-ac65-f86b9b848e77 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:26.224878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ac53-4253-44c2-ac65-f86b9b848e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:26.225174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ac53-4253-44c2-ac65-f86b9b848e77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:26.226006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4ac53-4253-44c2-ac65-f86b9b848e77 None admin] RBAC: Adding query filter params (role.id=41fd0da9b16d41c08992073473d87488, scope.project.id=f09928d8e8054beca93d734e72641b8e, user.id=250d4c8e98574ef29b5672702970f305) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:26.226320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4ac53-4253-44c2-ac65-f86b9b848e77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:26.227787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4ac53-4253-44c2-ac65-f86b9b848e77 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:26.233501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 568/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:25:26 2022] GET /identity/v3/role_assignments?user.id=250d4c8e98574ef29b5672702970f305&scope.project.id=f09928d8e8054beca93d734e72641b8e&role.id=41fd0da9b16d41c08992073473d87488 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:28.250805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bb125-ab4c-4632-9e15-675c4c59fd7e None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:28.251313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bb125-ab4c-4632-9e15-675c4c59fd7e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:28.251313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bb125-ab4c-4632-9e15-675c4c59fd7e None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:28.252655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 563/1131] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:28.258070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdb22c-a51e-44f8-8f7b-e22e17ee6885 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:28.258472 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdb22c-a51e-44f8-8f7b-e22e17ee6885 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:28.258636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdb22c-a51e-44f8-8f7b-e22e17ee6885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:28.275377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3afdb22c-a51e-44f8-8f7b-e22e17ee6885 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:28.276571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3afdb22c-a51e-44f8-8f7b-e22e17ee6885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:28.321232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:28.340399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 569/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:28.364016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bb125-ab4c-4632-9e15-675c4c59fd7e None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:28.464188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:28.473850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bb125-ab4c-4632-9e15-675c4c59fd7e None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:28.507315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863798cb-4ffc-4499-a66c-082e81db679f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:28.508105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863798cb-4ffc-4499-a66c-082e81db679f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:28.508391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863798cb-4ffc-4499-a66c-082e81db679f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:28.508669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863798cb-4ffc-4499-a66c-082e81db679f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:28.509838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863798cb-4ffc-4499-a66c-082e81db679f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:28.511264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863798cb-4ffc-4499-a66c-082e81db679f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:28.512871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 564/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:28.519529 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdb22c-a51e-44f8-8f7b-e22e17ee6885 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:28.613474 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdb22c-a51e-44f8-8f7b-e22e17ee6885 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:28.646822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea594c20-e708-4f2b-b5b9-0cacd169b2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:28.647476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea594c20-e708-4f2b-b5b9-0cacd169b2e9 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:28.647759 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea594c20-e708-4f2b-b5b9-0cacd169b2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:28.648035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea594c20-e708-4f2b-b5b9-0cacd169b2e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:28.648744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea594c20-e708-4f2b-b5b9-0cacd169b2e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:28.650193 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea594c20-e708-4f2b-b5b9-0cacd169b2e9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:28.764500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 570/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:25:28 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:30.846608 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306e2ae-1dd9-47af-90e5-022c7574aab4 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:30.847113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306e2ae-1dd9-47af-90e5-022c7574aab4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:30.847302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306e2ae-1dd9-47af-90e5-022c7574aab4 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:30.848517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 565/1135] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:25:30.853756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a32af7-37cc-453d-93d5-8fe283645ba5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:30.854204 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a32af7-37cc-453d-93d5-8fe283645ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:30.854299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a32af7-37cc-453d-93d5-8fe283645ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:30.871525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-84a32af7-37cc-453d-93d5-8fe283645ba5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:30.872701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-84a32af7-37cc-453d-93d5-8fe283645ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:30.901530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:30.940915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 571/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:25:30.966313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306e2ae-1dd9-47af-90e5-022c7574aab4 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:31.070766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306e2ae-1dd9-47af-90e5-022c7574aab4 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:31.111415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0da28-75bc-46fd-bc03-c9ce3e493f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:31.112191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0da28-75bc-46fd-bc03-c9ce3e493f5a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:31.112478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0da28-75bc-46fd-bc03-c9ce3e493f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:31.112753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0da28-75bc-46fd-bc03-c9ce3e493f5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:31.113963 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0da28-75bc-46fd-bc03-c9ce3e493f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:31.115768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0da28-75bc-46fd-bc03-c9ce3e493f5a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:31.118462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 566/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:25:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:31.124213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a32af7-37cc-453d-93d5-8fe283645ba5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:31.219582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a32af7-37cc-453d-93d5-8fe283645ba5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:31.252617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41d7c5-d0ad-4675-a288-bc3fc745b7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:31.253318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41d7c5-d0ad-4675-a288-bc3fc745b7d4 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:31.253601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41d7c5-d0ad-4675-a288-bc3fc745b7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:31.253874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41d7c5-d0ad-4675-a288-bc3fc745b7d4 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:31.254573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41d7c5-d0ad-4675-a288-bc3fc745b7d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:31.255992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41d7c5-d0ad-4675-a288-bc3fc745b7d4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:31.362278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 572/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:25:31 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:33.458177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780bc1a-7923-4d61-938a-2ea5a2d60528 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:33.458708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780bc1a-7923-4d61-938a-2ea5a2d60528 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:33.458708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780bc1a-7923-4d61-938a-2ea5a2d60528 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:33.460087 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 567/1139] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:33.465254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38954c2d-311c-4a94-8ab6-b665929bb358 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:33.465692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38954c2d-311c-4a94-8ab6-b665929bb358 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:33.465787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38954c2d-311c-4a94-8ab6-b665929bb358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:33.482703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-38954c2d-311c-4a94-8ab6-b665929bb358 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:33.483876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-38954c2d-311c-4a94-8ab6-b665929bb358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:33.497590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:33.548736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 573/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:33.572918 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780bc1a-7923-4d61-938a-2ea5a2d60528 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:33.602144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:33.673282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780bc1a-7923-4d61-938a-2ea5a2d60528 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:33.706726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c68558-e061-4143-a1ed-0bcb9577e883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:33.707499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c68558-e061-4143-a1ed-0bcb9577e883 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:33.707779 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c68558-e061-4143-a1ed-0bcb9577e883 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:33.708052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c68558-e061-4143-a1ed-0bcb9577e883 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:33.718012 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c68558-e061-4143-a1ed-0bcb9577e883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-37c68558-e061-4143-a1ed-0bcb9577e883 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:33.725654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:33.726878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:33.727998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 568/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:25:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:33.733006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38954c2d-311c-4a94-8ab6-b665929bb358 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:33.827818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38954c2d-311c-4a94-8ab6-b665929bb358 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:33.860554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd83247-2a77-415f-809d-8df7f2ae84ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:33.861234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd83247-2a77-415f-809d-8df7f2ae84ab None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:33.861518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd83247-2a77-415f-809d-8df7f2ae84ab None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:33.861793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd83247-2a77-415f-809d-8df7f2ae84ab None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:33.862549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd83247-2a77-415f-809d-8df7f2ae84ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:33.862847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd83247-2a77-415f-809d-8df7f2ae84ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:33.864222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd83247-2a77-415f-809d-8df7f2ae84ab None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:33.872414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 574/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:25:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:33.878850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c68558-e061-4143-a1ed-0bcb9577e883 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:33.979858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:33.991966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c68558-e061-4143-a1ed-0bcb9577e883 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:34.025581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad53c3-9cde-4719-9854-360c5ace2e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:34.026437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad53c3-9cde-4719-9854-360c5ace2e1a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:34.026728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad53c3-9cde-4719-9854-360c5ace2e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:34.027001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad53c3-9cde-4719-9854-360c5ace2e1a None admin] PATH_INFO: `/v3/users/a6480c5896b246e7b773652fd7cae6ed` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:34.040516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad53c3-9cde-4719-9854-360c5ace2e1a None admin] RBAC: Authorizing `identity:get_user(user_id=a6480c5896b246e7b773652fd7cae6ed)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:34.041997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad53c3-9cde-4719-9854-360c5ace2e1a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:34.043674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 569/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:33 2022] GET /identity/v3/users/a6480c5896b246e7b773652fd7cae6ed => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:34.050251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd83247-2a77-415f-809d-8df7f2ae84ab None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:34.066040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:34.149402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd83247-2a77-415f-809d-8df7f2ae84ab None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:34.185094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fd506-34c9-45ff-8f7a-96aac2f0c04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:34.185785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fd506-34c9-45ff-8f7a-96aac2f0c04c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:34.186068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fd506-34c9-45ff-8f7a-96aac2f0c04c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:34.186344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fd506-34c9-45ff-8f7a-96aac2f0c04c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:34.254502 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fd506-34c9-45ff-8f7a-96aac2f0c04c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:34.255919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fd506-34c9-45ff-8f7a-96aac2f0c04c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:34.271353 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:34.355064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 575/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:25:34.362014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad53c3-9cde-4719-9854-360c5ace2e1a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:34.464139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad53c3-9cde-4719-9854-360c5ace2e1a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:34.496753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1aeff0-fc2d-490f-a85b-08f541d79624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:34.497627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aeff0-fc2d-490f-a85b-08f541d79624 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:34.497907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aeff0-fc2d-490f-a85b-08f541d79624 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:34.498178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1aeff0-fc2d-490f-a85b-08f541d79624 None admin] PATH_INFO: `/v3/projects/8a994cab73de421185ee2ca5c6c1450e` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:34.499342 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1aeff0-fc2d-490f-a85b-08f541d79624 None admin] RBAC: Authorizing `identity:get_project(project_id=8a994cab73de421185ee2ca5c6c1450e)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:34.501328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1aeff0-fc2d-490f-a85b-08f541d79624 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:34.502998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 570/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:34 2022] GET /identity/v3/projects/8a994cab73de421185ee2ca5c6c1450e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:34.509314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fd506-34c9-45ff-8f7a-96aac2f0c04c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:34.604913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fd506-34c9-45ff-8f7a-96aac2f0c04c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:34.617533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:34.637829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713bd70-303c-45c6-856b-b8381f3f7cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:34.639167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713bd70-303c-45c6-856b-b8381f3f7cbe None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:34.639167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713bd70-303c-45c6-856b-b8381f3f7cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:34.639167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713bd70-303c-45c6-856b-b8381f3f7cbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:34.639810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713bd70-303c-45c6-856b-b8381f3f7cbe None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=8a994cab73de421185ee2ca5c6c1450e, user.id=a6480c5896b246e7b773652fd7cae6ed) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:34.640120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713bd70-303c-45c6-856b-b8381f3f7cbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:34.642223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713bd70-303c-45c6-856b-b8381f3f7cbe None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:34.647528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 576/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:25:34 2022] GET /identity/v3/role_assignments?user.id=a6480c5896b246e7b773652fd7cae6ed&scope.project.id=8a994cab73de421185ee2ca5c6c1450e&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:36.657651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75468b-16fb-4069-83a0-6f0ff57bf515 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:36.658239 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75468b-16fb-4069-83a0-6f0ff57bf515 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:36.658239 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75468b-16fb-4069-83a0-6f0ff57bf515 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:36.659553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 571/1147] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:36.664931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9084f17-a0d0-4b50-930e-afb8255b4272 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:36.665401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9084f17-a0d0-4b50-930e-afb8255b4272 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:36.665492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9084f17-a0d0-4b50-930e-afb8255b4272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:36.684908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e9084f17-a0d0-4b50-930e-afb8255b4272 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:36.688296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e9084f17-a0d0-4b50-930e-afb8255b4272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:36.762735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 577/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:36.786284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75468b-16fb-4069-83a0-6f0ff57bf515 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:36.802656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:36.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75468b-16fb-4069-83a0-6f0ff57bf515 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:36.925421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec35e3d-8e3d-4179-a7dc-b5e791a6de67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:36.926123 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec35e3d-8e3d-4179-a7dc-b5e791a6de67 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:36.926404 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec35e3d-8e3d-4179-a7dc-b5e791a6de67 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:36.926673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec35e3d-8e3d-4179-a7dc-b5e791a6de67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:36.938559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec35e3d-8e3d-4179-a7dc-b5e791a6de67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-0ec35e3d-8e3d-4179-a7dc-b5e791a6de67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:36.946763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:36.956356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:36.957418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 572/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:25:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:36.959238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9084f17-a0d0-4b50-930e-afb8255b4272 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:37.053146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9084f17-a0d0-4b50-930e-afb8255b4272 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:37.060672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:37.093492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e007bc0-4140-440c-81c7-853fd7c6d45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:37.096020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e007bc0-4140-440c-81c7-853fd7c6d45c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:37.096020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e007bc0-4140-440c-81c7-853fd7c6d45c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:37.096020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e007bc0-4140-440c-81c7-853fd7c6d45c None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:37.096020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e007bc0-4140-440c-81c7-853fd7c6d45c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:37.096020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e007bc0-4140-440c-81c7-853fd7c6d45c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:37.097319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e007bc0-4140-440c-81c7-853fd7c6d45c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:37.105446 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 578/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:25:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:37.112099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec35e3d-8e3d-4179-a7dc-b5e791a6de67 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:37.187975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:37.214827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec35e3d-8e3d-4179-a7dc-b5e791a6de67 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:37.251669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c8b43-1088-4c07-943e-152334ef2072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:37.252534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c8b43-1088-4c07-943e-152334ef2072 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:37.252819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c8b43-1088-4c07-943e-152334ef2072 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:37.253095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c8b43-1088-4c07-943e-152334ef2072 None admin] PATH_INFO: `/v3/users/a6480c5896b246e7b773652fd7cae6ed` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:37.254321 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c8b43-1088-4c07-943e-152334ef2072 None admin] RBAC: Authorizing `identity:get_user(user_id=a6480c5896b246e7b773652fd7cae6ed)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:37.255763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c8b43-1088-4c07-943e-152334ef2072 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:37.257430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 573/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:37 2022] GET /identity/v3/users/a6480c5896b246e7b773652fd7cae6ed => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:37.263412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e007bc0-4140-440c-81c7-853fd7c6d45c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:37.363277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e007bc0-4140-440c-81c7-853fd7c6d45c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:37.398010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ba0f7-5880-42cb-9e54-272e69ecf2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:37.398875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba0f7-5880-42cb-9e54-272e69ecf2ad None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:37.399154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba0f7-5880-42cb-9e54-272e69ecf2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:37.399427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba0f7-5880-42cb-9e54-272e69ecf2ad None admin] PATH_INFO: `/v3/projects/8a994cab73de421185ee2ca5c6c1450e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:37.400608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ba0f7-5880-42cb-9e54-272e69ecf2ad None admin] RBAC: Authorizing `identity:get_project(project_id=8a994cab73de421185ee2ca5c6c1450e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:37.402062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ba0f7-5880-42cb-9e54-272e69ecf2ad None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:37.403934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 579/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:37 2022] GET /identity/v3/projects/8a994cab73de421185ee2ca5c6c1450e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:37.410363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c8b43-1088-4c07-943e-152334ef2072 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:37.514624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c8b43-1088-4c07-943e-152334ef2072 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:37.548588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b2789-ba13-4738-8177-9302a527b9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:37.549498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b2789-ba13-4738-8177-9302a527b9a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:37.549786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b2789-ba13-4738-8177-9302a527b9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:37.550064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b2789-ba13-4738-8177-9302a527b9a7 None admin] PATH_INFO: `/v3/projects/8a994cab73de421185ee2ca5c6c1450e/users/a6480c5896b246e7b773652fd7cae6ed/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:37.552019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b2789-ba13-4738-8177-9302a527b9a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a994cab73de421185ee2ca5c6c1450e, user_id=a6480c5896b246e7b773652fd7cae6ed, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:37.553494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b2789-ba13-4738-8177-9302a527b9a7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:37.685267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 574/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:25:37 2022] PUT /identity/v3/projects/8a994cab73de421185ee2ca5c6c1450e/users/a6480c5896b246e7b773652fd7cae6ed/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:25:39.706066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b18e4-efd1-4621-84a7-b68f1ba59e22 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:39.706620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b18e4-efd1-4621-84a7-b68f1ba59e22 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:39.706620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b18e4-efd1-4621-84a7-b68f1ba59e22 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:39.707956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 580/1154] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:39.713106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0acd-fab4-4a03-b19d-c77e2c8ea9fa None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:39.713539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0acd-fab4-4a03-b19d-c77e2c8ea9fa None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:39.713690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0acd-fab4-4a03-b19d-c77e2c8ea9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:39.718617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:39.731240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-143d0acd-fab4-4a03-b19d-c77e2c8ea9fa None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:39.732758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-143d0acd-fab4-4a03-b19d-c77e2c8ea9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:39.799095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 575/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:39.821948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b18e4-efd1-4621-84a7-b68f1ba59e22 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:39.857368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:39.920063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b18e4-efd1-4621-84a7-b68f1ba59e22 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:39.954295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31e0df-c36d-44bd-916c-486c4be13877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:39.955080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31e0df-c36d-44bd-916c-486c4be13877 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:39.955363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31e0df-c36d-44bd-916c-486c4be13877 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:39.955637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31e0df-c36d-44bd-916c-486c4be13877 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:39.965712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31e0df-c36d-44bd-916c-486c4be13877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-bc31e0df-c36d-44bd-916c-486c4be13877 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:39.973805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:39.974966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:39.976195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 581/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:25:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:39.981091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d0acd-fab4-4a03-b19d-c77e2c8ea9fa None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:40.077996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d0acd-fab4-4a03-b19d-c77e2c8ea9fa None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:40.107316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:40.112039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874de4e5-150c-4686-8ba3-6bc14e450046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:40.112700 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874de4e5-150c-4686-8ba3-6bc14e450046 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:40.112985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874de4e5-150c-4686-8ba3-6bc14e450046 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:40.113294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874de4e5-150c-4686-8ba3-6bc14e450046 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:40.114048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874de4e5-150c-4686-8ba3-6bc14e450046 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:40.122056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874de4e5-150c-4686-8ba3-6bc14e450046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:40.122056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874de4e5-150c-4686-8ba3-6bc14e450046 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:40.123485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 576/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:25:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:40.129494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31e0df-c36d-44bd-916c-486c4be13877 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:40.221207 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:40.226679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31e0df-c36d-44bd-916c-486c4be13877 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:40.258660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9864510-72c1-4bbc-870b-0f859058425c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:40.259517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9864510-72c1-4bbc-870b-0f859058425c None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:40.259799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9864510-72c1-4bbc-870b-0f859058425c None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:40.260074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9864510-72c1-4bbc-870b-0f859058425c None admin] PATH_INFO: `/v3/users/a6480c5896b246e7b773652fd7cae6ed` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:40.261300 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9864510-72c1-4bbc-870b-0f859058425c None admin] RBAC: Authorizing `identity:get_user(user_id=a6480c5896b246e7b773652fd7cae6ed)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:40.262726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9864510-72c1-4bbc-870b-0f859058425c None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:40.264345 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 582/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:40 2022] GET /identity/v3/users/a6480c5896b246e7b773652fd7cae6ed => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:40.270523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874de4e5-150c-4686-8ba3-6bc14e450046 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:40.367084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874de4e5-150c-4686-8ba3-6bc14e450046 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:40.400256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c99ab2-6351-4e48-88a4-0eaf90a2b353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:40.400904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c99ab2-6351-4e48-88a4-0eaf90a2b353 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:40.401206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c99ab2-6351-4e48-88a4-0eaf90a2b353 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:40.401488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c99ab2-6351-4e48-88a4-0eaf90a2b353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:40.462774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:40.470057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c99ab2-6351-4e48-88a4-0eaf90a2b353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:40.508627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c99ab2-6351-4e48-88a4-0eaf90a2b353 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:40.565857 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 577/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:25:40.571867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9864510-72c1-4bbc-870b-0f859058425c None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:40.667154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9864510-72c1-4bbc-870b-0f859058425c None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:40.700132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3132d-bd2b-4b4a-9c1b-970b8cd9c513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:40.700970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3132d-bd2b-4b4a-9c1b-970b8cd9c513 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:40.701280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3132d-bd2b-4b4a-9c1b-970b8cd9c513 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:40.701556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3132d-bd2b-4b4a-9c1b-970b8cd9c513 None admin] PATH_INFO: `/v3/projects/8a994cab73de421185ee2ca5c6c1450e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:40.702731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3132d-bd2b-4b4a-9c1b-970b8cd9c513 None admin] RBAC: Authorizing `identity:get_project(project_id=8a994cab73de421185ee2ca5c6c1450e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:40.705055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3132d-bd2b-4b4a-9c1b-970b8cd9c513 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:40.706846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 583/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:40 2022] GET /identity/v3/projects/8a994cab73de421185ee2ca5c6c1450e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:40.712896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c99ab2-6351-4e48-88a4-0eaf90a2b353 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:40.802385 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:40.808892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c99ab2-6351-4e48-88a4-0eaf90a2b353 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:40.843307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27340bdf-ec12-4743-ba24-0649980a5795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:40.843978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27340bdf-ec12-4743-ba24-0649980a5795 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:40.844263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27340bdf-ec12-4743-ba24-0649980a5795 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:40.844539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27340bdf-ec12-4743-ba24-0649980a5795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:40.845401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27340bdf-ec12-4743-ba24-0649980a5795 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=8a994cab73de421185ee2ca5c6c1450e, user.id=a6480c5896b246e7b773652fd7cae6ed) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:40.845714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27340bdf-ec12-4743-ba24-0649980a5795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:40.847155 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27340bdf-ec12-4743-ba24-0649980a5795 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:40.852850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 578/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:25:40 2022] GET /identity/v3/role_assignments?user.id=a6480c5896b246e7b773652fd7cae6ed&scope.project.id=8a994cab73de421185ee2ca5c6c1450e&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:42.877069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4f7ff-bb7b-43c1-aac1-8fd47a9bc689 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:42.877646 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4f7ff-bb7b-43c1-aac1-8fd47a9bc689 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:42.877646 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4f7ff-bb7b-43c1-aac1-8fd47a9bc689 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:42.879033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 584/1162] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:42.884280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64bf0b-9a22-47aa-ab70-3ffb7bd7a28a None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:42.884689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64bf0b-9a22-47aa-ab70-3ffb7bd7a28a None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:42.884842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64bf0b-9a22-47aa-ab70-3ffb7bd7a28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:42.901183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-bc64bf0b-9a22-47aa-ab70-3ffb7bd7a28a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:42.902371 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-bc64bf0b-9a22-47aa-ab70-3ffb7bd7a28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:42.966560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 579/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:42.989989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4f7ff-bb7b-43c1-aac1-8fd47a9bc689 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:43.021585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:43.096795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4f7ff-bb7b-43c1-aac1-8fd47a9bc689 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:43.130130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbe544-8d7c-4845-b25d-cf72705aeccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:43.130802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbe544-8d7c-4845-b25d-cf72705aeccd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:43.131088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbe544-8d7c-4845-b25d-cf72705aeccd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:43.131365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbe544-8d7c-4845-b25d-cf72705aeccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:43.197514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbe544-8d7c-4845-b25d-cf72705aeccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:43.198951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbe544-8d7c-4845-b25d-cf72705aeccd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:43.228652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:43.295153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 585/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:25:43.301397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64bf0b-9a22-47aa-ab70-3ffb7bd7a28a None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:43.386088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:43.399398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64bf0b-9a22-47aa-ab70-3ffb7bd7a28a None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:43.442876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0480579-9e0e-4f42-a1a9-858ed9f9ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:43.443709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0480579-9e0e-4f42-a1a9-858ed9f9ad68 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:43.444008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0480579-9e0e-4f42-a1a9-858ed9f9ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:43.444302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0480579-9e0e-4f42-a1a9-858ed9f9ad68 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:43.451592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0480579-9e0e-4f42-a1a9-858ed9f9ad68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:43.453147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0480579-9e0e-4f42-a1a9-858ed9f9ad68 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-a0480579-9e0e-4f42-a1a9-858ed9f9ad68 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:43.467704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:43.468969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:43.470186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 580/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jul 14 11:25:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:43.475233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbe544-8d7c-4845-b25d-cf72705aeccd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:43.572946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbe544-8d7c-4845-b25d-cf72705aeccd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:43.600808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:43.605484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32546ca4-ddb8-42db-b6df-19a05bf0e9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:43.606148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32546ca4-ddb8-42db-b6df-19a05bf0e9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:43.606421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32546ca4-ddb8-42db-b6df-19a05bf0e9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:43.606694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32546ca4-ddb8-42db-b6df-19a05bf0e9e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:43.607415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32546ca4-ddb8-42db-b6df-19a05bf0e9e5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:43.607713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32546ca4-ddb8-42db-b6df-19a05bf0e9e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:43.609071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32546ca4-ddb8-42db-b6df-19a05bf0e9e5 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:43.616167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 586/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jul 14 11:25:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:45.619572 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cceab7a-f1e9-4413-b675-7bd22884dc30 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:45.620115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cceab7a-f1e9-4413-b675-7bd22884dc30 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:45.620115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cceab7a-f1e9-4413-b675-7bd22884dc30 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:45.621523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 581/1167] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:45.627348 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c37232-1f12-4912-b28e-8ef3539e029a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:45.627772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c37232-1f12-4912-b28e-8ef3539e029a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:45.627911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c37232-1f12-4912-b28e-8ef3539e029a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:45.644963 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c3c37232-1f12-4912-b28e-8ef3539e029a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:45.646196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c3c37232-1f12-4912-b28e-8ef3539e029a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:45.711086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 587/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:45.733596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cceab7a-f1e9-4413-b675-7bd22884dc30 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:45.833746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cceab7a-f1e9-4413-b675-7bd22884dc30 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:45.865911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98158c25-16ce-4051-ad78-46d499fbba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:45.866562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158c25-16ce-4051-ad78-46d499fbba68 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:45.866847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158c25-16ce-4051-ad78-46d499fbba68 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:45.867122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158c25-16ce-4051-ad78-46d499fbba68 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:45.867878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98158c25-16ce-4051-ad78-46d499fbba68 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:45.869288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98158c25-16ce-4051-ad78-46d499fbba68 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:45.956741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 582/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Thu Jul 14 11:25:45 2022] POST /identity/v3/domains => generated 267 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:47.998352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986abe4e-c134-48d0-9d34-90b022b4c687 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:47.998880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986abe4e-c134-48d0-9d34-90b022b4c687 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:47.999007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986abe4e-c134-48d0-9d34-90b022b4c687 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:48.001116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 588/1170] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:25:48.006575 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd5f16-47e9-4ce7-8c85-bf5614531770 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:48.007002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd5f16-47e9-4ce7-8c85-bf5614531770 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:48.007131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd5f16-47e9-4ce7-8c85-bf5614531770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:48.024122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-d9bd5f16-47e9-4ce7-8c85-bf5614531770 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:48.025337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d9bd5f16-47e9-4ce7-8c85-bf5614531770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:48.027264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:48.090040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 583/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:48.113465 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986abe4e-c134-48d0-9d34-90b022b4c687 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:48.212634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986abe4e-c134-48d0-9d34-90b022b4c687 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:48.238170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:48.246378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14df51c-61c0-4888-8154-7ecacff0afc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:48.247239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df51c-61c0-4888-8154-7ecacff0afc4 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:48.252934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df51c-61c0-4888-8154-7ecacff0afc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:48.252934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df51c-61c0-4888-8154-7ecacff0afc4 None admin] PATH_INFO: `/v3/domains/b62e9c490d5a4905b9c40da9c394622e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:48.258317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14df51c-61c0-4888-8154-7ecacff0afc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b62e9c490d5a4905b9c40da9c394622e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:48.259761 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14df51c-61c0-4888-8154-7ecacff0afc4 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:48.261477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 589/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Thu Jul 14 11:25:48 2022] GET /identity/v3/domains/b62e9c490d5a4905b9c40da9c394622e => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:25:48.267979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd5f16-47e9-4ce7-8c85-bf5614531770 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:48.362778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd5f16-47e9-4ce7-8c85-bf5614531770 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:48.399674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc3e2e-fec1-4b2d-9406-9513b3489a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:48.400335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc3e2e-fec1-4b2d-9406-9513b3489a92 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:48.400616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc3e2e-fec1-4b2d-9406-9513b3489a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:48.400887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc3e2e-fec1-4b2d-9406-9513b3489a92 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:48.401633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc3e2e-fec1-4b2d-9406-9513b3489a92 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:48.403069 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc3e2e-fec1-4b2d-9406-9513b3489a92 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:48.411668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:48.497649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 584/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:25:48 2022] POST /identity/v3/projects => generated 361 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:50.548888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c42b2e-9eaf-4342-890a-7a861d9295f5 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:50.549612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c42b2e-9eaf-4342-890a-7a861d9295f5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:50.549770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c42b2e-9eaf-4342-890a-7a861d9295f5 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:50.551893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 590/1174] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:50.557336 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab2ffc-6ca0-44c4-b87a-a685ceec776d None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:50.557729 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab2ffc-6ca0-44c4-b87a-a685ceec776d None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:50.557892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab2ffc-6ca0-44c4-b87a-a685ceec776d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:50.575058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-eaab2ffc-6ca0-44c4-b87a-a685ceec776d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:50.576235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-eaab2ffc-6ca0-44c4-b87a-a685ceec776d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:50.641228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 585/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:50.665830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c42b2e-9eaf-4342-890a-7a861d9295f5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:50.763646 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c42b2e-9eaf-4342-890a-7a861d9295f5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:50.797028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567064c-70e7-41fc-9304-12e2a116d4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:50.797924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567064c-70e7-41fc-9304-12e2a116d4cc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:50.798209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567064c-70e7-41fc-9304-12e2a116d4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:50.798496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567064c-70e7-41fc-9304-12e2a116d4cc None admin] PATH_INFO: `/v3/domains/b62e9c490d5a4905b9c40da9c394622e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:50.799739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567064c-70e7-41fc-9304-12e2a116d4cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=b62e9c490d5a4905b9c40da9c394622e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:50.801128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567064c-70e7-41fc-9304-12e2a116d4cc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:50.802771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 591/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Thu Jul 14 11:25:50 2022] GET /identity/v3/domains/b62e9c490d5a4905b9c40da9c394622e => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:50.808427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab2ffc-6ca0-44c4-b87a-a685ceec776d None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:50.919966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab2ffc-6ca0-44c4-b87a-a685ceec776d None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:50.948644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:50.964185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09417a-5ac5-4a5a-a939-14cc4ac6ae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:50.964857 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09417a-5ac5-4a5a-a939-14cc4ac6ae40 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:50.965143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09417a-5ac5-4a5a-a939-14cc4ac6ae40 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:50.965451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09417a-5ac5-4a5a-a939-14cc4ac6ae40 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:50.966183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09417a-5ac5-4a5a-a939-14cc4ac6ae40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:50.967627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09417a-5ac5-4a5a-a939-14cc4ac6ae40 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:51.057232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 586/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:25:50 2022] POST /identity/v3/users => generated 311 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:53.197851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14329bc-06e7-4284-a95e-8f4426cdd834 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:53.198397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14329bc-06e7-4284-a95e-8f4426cdd834 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:53.198397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14329bc-06e7-4284-a95e-8f4426cdd834 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:53.199697 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 592/1178] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:53.204970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91dab5-52ee-4b8e-98ab-05ff5fcdad42 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:53.205445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91dab5-52ee-4b8e-98ab-05ff5fcdad42 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:53.205545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91dab5-52ee-4b8e-98ab-05ff5fcdad42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:53.222449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-bc91dab5-52ee-4b8e-98ab-05ff5fcdad42 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:53.223633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-bc91dab5-52ee-4b8e-98ab-05ff5fcdad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:53.288205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 587/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:53.311817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14329bc-06e7-4284-a95e-8f4426cdd834 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:53.366660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:53.411288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14329bc-06e7-4284-a95e-8f4426cdd834 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:53.445270 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be982252-a7e1-4cf8-82cb-d17fe49a8fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:53.446055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be982252-a7e1-4cf8-82cb-d17fe49a8fcb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:53.446339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be982252-a7e1-4cf8-82cb-d17fe49a8fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:53.446870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be982252-a7e1-4cf8-82cb-d17fe49a8fcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:53.456995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be982252-a7e1-4cf8-82cb-d17fe49a8fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-be982252-a7e1-4cf8-82cb-d17fe49a8fcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:53.464523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:53.465726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:53.466841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 593/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:25:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:53.472332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91dab5-52ee-4b8e-98ab-05ff5fcdad42 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:53.568200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91dab5-52ee-4b8e-98ab-05ff5fcdad42 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:53.575597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:53.603048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d9b32-3093-40d9-8b82-397c987853f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:53.603700 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d9b32-3093-40d9-8b82-397c987853f8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:53.603983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d9b32-3093-40d9-8b82-397c987853f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:53.604327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d9b32-3093-40d9-8b82-397c987853f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:53.605313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d9b32-3093-40d9-8b82-397c987853f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:53.605846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d9b32-3093-40d9-8b82-397c987853f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:53.607679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d9b32-3093-40d9-8b82-397c987853f8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:53.615830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 588/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:53.621825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be982252-a7e1-4cf8-82cb-d17fe49a8fcb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:53.719009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be982252-a7e1-4cf8-82cb-d17fe49a8fcb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:53.731794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:53.751584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63b859-9df0-4082-b551-a1e60cc33eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:53.752457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63b859-9df0-4082-b551-a1e60cc33eff None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:53.752741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63b859-9df0-4082-b551-a1e60cc33eff None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:53.753046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63b859-9df0-4082-b551-a1e60cc33eff None admin] PATH_INFO: `/v3/users/86fbe4a55e3847529eb22ff58ee75c7b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:53.767761 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63b859-9df0-4082-b551-a1e60cc33eff None admin] RBAC: Authorizing `identity:get_user(user_id=86fbe4a55e3847529eb22ff58ee75c7b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:53.769233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63b859-9df0-4082-b551-a1e60cc33eff None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:53.771393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 594/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:53 2022] GET /identity/v3/users/86fbe4a55e3847529eb22ff58ee75c7b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:53.776930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d9b32-3093-40d9-8b82-397c987853f8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:53.876235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d9b32-3093-40d9-8b82-397c987853f8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:53.909749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd2e52-3221-422b-8356-35e12dc1035a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:53.910388 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd2e52-3221-422b-8356-35e12dc1035a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:53.910674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd2e52-3221-422b-8356-35e12dc1035a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:53.910953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd2e52-3221-422b-8356-35e12dc1035a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:53.944723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:53.977726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd2e52-3221-422b-8356-35e12dc1035a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:53.979118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd2e52-3221-422b-8356-35e12dc1035a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:54.080037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 589/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:25:54.086005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63b859-9df0-4082-b551-a1e60cc33eff None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:54.182837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63b859-9df0-4082-b551-a1e60cc33eff None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:54.219334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecad835-2388-4e43-8806-e9aa4caf5bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:54.220187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecad835-2388-4e43-8806-e9aa4caf5bec None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:54.220468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecad835-2388-4e43-8806-e9aa4caf5bec None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:54.220737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecad835-2388-4e43-8806-e9aa4caf5bec None admin] PATH_INFO: `/v3/projects/e723438c25614a6babae92c8e882f147` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:54.221940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecad835-2388-4e43-8806-e9aa4caf5bec None admin] RBAC: Authorizing `identity:get_project(project_id=e723438c25614a6babae92c8e882f147)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:54.223363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecad835-2388-4e43-8806-e9aa4caf5bec None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:54.225075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 595/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:54 2022] GET /identity/v3/projects/e723438c25614a6babae92c8e882f147 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:54.231206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd2e52-3221-422b-8356-35e12dc1035a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:54.296055 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:54.327305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd2e52-3221-422b-8356-35e12dc1035a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:54.359901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31441e-1669-4916-bca6-11dcaf6d68fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:54.360536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31441e-1669-4916-bca6-11dcaf6d68fc None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:54.360814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31441e-1669-4916-bca6-11dcaf6d68fc None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:54.361085 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31441e-1669-4916-bca6-11dcaf6d68fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:54.361937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31441e-1669-4916-bca6-11dcaf6d68fc None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=e723438c25614a6babae92c8e882f147, user.id=86fbe4a55e3847529eb22ff58ee75c7b) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:54.362246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31441e-1669-4916-bca6-11dcaf6d68fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:54.364066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31441e-1669-4916-bca6-11dcaf6d68fc None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:54.368936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 590/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:25:54 2022] GET /identity/v3/role_assignments?user.id=86fbe4a55e3847529eb22ff58ee75c7b&scope.project.id=e723438c25614a6babae92c8e882f147&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:56.364843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aa85b-4679-41e1-a4cf-eda89648e503 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:56.365438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aa85b-4679-41e1-a4cf-eda89648e503 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:56.365438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aa85b-4679-41e1-a4cf-eda89648e503 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:56.366787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 596/1186] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:56.372178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530b02-6767-4da0-83a0-1d4049ea1b19 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:56.372622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530b02-6767-4da0-83a0-1d4049ea1b19 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:56.372726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530b02-6767-4da0-83a0-1d4049ea1b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:56.389617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-1d530b02-6767-4da0-83a0-1d4049ea1b19 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:56.391479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1d530b02-6767-4da0-83a0-1d4049ea1b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:56.457923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 591/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:56.480371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1aa85b-4679-41e1-a4cf-eda89648e503 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:56.524877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:56.581751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1aa85b-4679-41e1-a4cf-eda89648e503 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:56.614211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54b788-da7e-465d-9578-a2220553060d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:56.615034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54b788-da7e-465d-9578-a2220553060d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:56.615260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54b788-da7e-465d-9578-a2220553060d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:56.615527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54b788-da7e-465d-9578-a2220553060d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:56.625224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54b788-da7e-465d-9578-a2220553060d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-ec54b788-da7e-465d-9578-a2220553060d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:56.632748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:56.633898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:25:56.635037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 597/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:56.640845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530b02-6767-4da0-83a0-1d4049ea1b19 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:56.702165 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:56.739545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530b02-6767-4da0-83a0-1d4049ea1b19 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:56.772621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a799e-ad1b-4b7c-a052-6b298249d6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:56.773315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a799e-ad1b-4b7c-a052-6b298249d6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:56.773601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a799e-ad1b-4b7c-a052-6b298249d6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:56.773877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a799e-ad1b-4b7c-a052-6b298249d6f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:56.774626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a799e-ad1b-4b7c-a052-6b298249d6f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:56.774934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a799e-ad1b-4b7c-a052-6b298249d6f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:56.776313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a799e-ad1b-4b7c-a052-6b298249d6f5 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:56.785658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 592/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:25:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:56.791461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54b788-da7e-465d-9578-a2220553060d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:56.888490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:56.891701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54b788-da7e-465d-9578-a2220553060d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:56.926749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4427f02-f9aa-4e17-83d1-a8236fca4394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:56.927609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4427f02-f9aa-4e17-83d1-a8236fca4394 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:56.927890 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4427f02-f9aa-4e17-83d1-a8236fca4394 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:56.928164 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4427f02-f9aa-4e17-83d1-a8236fca4394 None admin] PATH_INFO: `/v3/users/86fbe4a55e3847529eb22ff58ee75c7b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:56.929361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4427f02-f9aa-4e17-83d1-a8236fca4394 None admin] RBAC: Authorizing `identity:get_user(user_id=86fbe4a55e3847529eb22ff58ee75c7b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:56.930787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4427f02-f9aa-4e17-83d1-a8236fca4394 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:56.932415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 598/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:56 2022] GET /identity/v3/users/86fbe4a55e3847529eb22ff58ee75c7b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:56.938438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a799e-ad1b-4b7c-a052-6b298249d6f5 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:57.034309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a799e-ad1b-4b7c-a052-6b298249d6f5 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:57.056934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:57.067241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62a4b4-b40a-4136-9f36-5b03acef1264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:57.069995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62a4b4-b40a-4136-9f36-5b03acef1264 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:57.069995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62a4b4-b40a-4136-9f36-5b03acef1264 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:57.069995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62a4b4-b40a-4136-9f36-5b03acef1264 None admin] PATH_INFO: `/v3/projects/e723438c25614a6babae92c8e882f147` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:57.069995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62a4b4-b40a-4136-9f36-5b03acef1264 None admin] RBAC: Authorizing `identity:get_project(project_id=e723438c25614a6babae92c8e882f147)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:57.071254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62a4b4-b40a-4136-9f36-5b03acef1264 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:57.073246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 593/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:25:56 2022] GET /identity/v3/projects/e723438c25614a6babae92c8e882f147 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:57.079170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4427f02-f9aa-4e17-83d1-a8236fca4394 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:57.180025 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4427f02-f9aa-4e17-83d1-a8236fca4394 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:57.212771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c7587-73ba-4316-b2b4-42891f00e915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:57.213685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c7587-73ba-4316-b2b4-42891f00e915 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:57.213972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c7587-73ba-4316-b2b4-42891f00e915 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:57.214245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c7587-73ba-4316-b2b4-42891f00e915 None admin] PATH_INFO: `/v3/projects/e723438c25614a6babae92c8e882f147/users/86fbe4a55e3847529eb22ff58ee75c7b/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:57.216189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c7587-73ba-4316-b2b4-42891f00e915 None admin] RBAC: Authorizing `identity:create_grant(project_id=e723438c25614a6babae92c8e882f147, user_id=86fbe4a55e3847529eb22ff58ee75c7b, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:57.217640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c7587-73ba-4316-b2b4-42891f00e915 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:57.303455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 599/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:25:57 2022] PUT /identity/v3/projects/e723438c25614a6babae92c8e882f147/users/86fbe4a55e3847529eb22ff58ee75c7b/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:25:59.359672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09ed18-7859-424a-936a-9315aa8f2a32 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:59.360232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09ed18-7859-424a-936a-9315aa8f2a32 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:59.360232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09ed18-7859-424a-936a-9315aa8f2a32 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:59.361625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 594/1193] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:25:59.366938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e02188-3df5-4e64-b634-06b09e5cc2b9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:59.367329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e02188-3df5-4e64-b634-06b09e5cc2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:59.367492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e02188-3df5-4e64-b634-06b09e5cc2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:59.384442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d6e02188-3df5-4e64-b634-06b09e5cc2b9 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:59.385651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d6e02188-3df5-4e64-b634-06b09e5cc2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:59.409093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:59.453470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 600/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:25:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:25:59.476781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09ed18-7859-424a-936a-9315aa8f2a32 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:59.578451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09ed18-7859-424a-936a-9315aa8f2a32 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:59.611286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe24afc-b186-4da2-9aec-0ada96247b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:59.611984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe24afc-b186-4da2-9aec-0ada96247b45 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:59.612265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe24afc-b186-4da2-9aec-0ada96247b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:59.612538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe24afc-b186-4da2-9aec-0ada96247b45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:59.622312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe24afc-b186-4da2-9aec-0ada96247b45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-bbe24afc-b186-4da2-9aec-0ada96247b45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:25:59.630381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:25:59.632048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:25:59.633127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 595/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:25:59.637599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e02188-3df5-4e64-b634-06b09e5cc2b9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:59.734617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e02188-3df5-4e64-b634-06b09e5cc2b9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:59.767911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394daab0-685d-48cc-b456-bbf3d9c792f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:59.768572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394daab0-685d-48cc-b456-bbf3d9c792f7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:59.768856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394daab0-685d-48cc-b456-bbf3d9c792f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:59.769128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394daab0-685d-48cc-b456-bbf3d9c792f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:59.769908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394daab0-685d-48cc-b456-bbf3d9c792f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:59.770215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394daab0-685d-48cc-b456-bbf3d9c792f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:59.771582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394daab0-685d-48cc-b456-bbf3d9c792f7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:59.779611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 601/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:59.785921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe24afc-b186-4da2-9aec-0ada96247b45 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:59.838872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:59.887530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe24afc-b186-4da2-9aec-0ada96247b45 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:59.921302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b263e-448f-46d3-916b-611fe1192540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:59.922164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b263e-448f-46d3-916b-611fe1192540 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:59.922450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b263e-448f-46d3-916b-611fe1192540 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:59.922727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b263e-448f-46d3-916b-611fe1192540 None admin] PATH_INFO: `/v3/users/86fbe4a55e3847529eb22ff58ee75c7b` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:59.923953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b263e-448f-46d3-916b-611fe1192540 None admin] RBAC: Authorizing `identity:get_user(user_id=86fbe4a55e3847529eb22ff58ee75c7b)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:59.925431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b263e-448f-46d3-916b-611fe1192540 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:59.927088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 596/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jul 14 11:25:59 2022] GET /identity/v3/users/86fbe4a55e3847529eb22ff58ee75c7b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:59.933336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394daab0-685d-48cc-b456-bbf3d9c792f7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:59.956901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:00.034750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394daab0-685d-48cc-b456-bbf3d9c792f7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:00.068697 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ade52-da55-4533-ae9f-91be856bcc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:00.069371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ade52-da55-4533-ae9f-91be856bcc6f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:00.069651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ade52-da55-4533-ae9f-91be856bcc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:00.069924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ade52-da55-4533-ae9f-91be856bcc6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:00.135064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ade52-da55-4533-ae9f-91be856bcc6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:00.136503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ade52-da55-4533-ae9f-91be856bcc6f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:00.156527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:00.231994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 602/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:25:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 11:26:00.238436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b263e-448f-46d3-916b-611fe1192540 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:00.341891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b263e-448f-46d3-916b-611fe1192540 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:00.362504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:00.378039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde952-92f4-4e39-b98d-5dcdddf36949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:00.378901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde952-92f4-4e39-b98d-5dcdddf36949 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:00.379193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde952-92f4-4e39-b98d-5dcdddf36949 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:00.379444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde952-92f4-4e39-b98d-5dcdddf36949 None admin] PATH_INFO: `/v3/projects/e723438c25614a6babae92c8e882f147` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:00.380636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde952-92f4-4e39-b98d-5dcdddf36949 None admin] RBAC: Authorizing `identity:get_project(project_id=e723438c25614a6babae92c8e882f147)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:00.382109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde952-92f4-4e39-b98d-5dcdddf36949 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:00.383808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 597/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jul 14 11:26:00 2022] GET /identity/v3/projects/e723438c25614a6babae92c8e882f147 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:26:00.390277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ade52-da55-4533-ae9f-91be856bcc6f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:00.491635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ade52-da55-4533-ae9f-91be856bcc6f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:00.516771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:00.530949 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74646b5c-db61-4e2f-9ea3-56f48ecb4a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:00.531606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74646b5c-db61-4e2f-9ea3-56f48ecb4a58 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:00.531885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74646b5c-db61-4e2f-9ea3-56f48ecb4a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:00.532153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74646b5c-db61-4e2f-9ea3-56f48ecb4a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:00.532982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74646b5c-db61-4e2f-9ea3-56f48ecb4a58 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=e723438c25614a6babae92c8e882f147, user.id=86fbe4a55e3847529eb22ff58ee75c7b) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:26:00.533324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74646b5c-db61-4e2f-9ea3-56f48ecb4a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:00.534760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74646b5c-db61-4e2f-9ea3-56f48ecb4a58 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:00.540583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 603/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:26:00 2022] GET /identity/v3/role_assignments?user.id=86fbe4a55e3847529eb22ff58ee75c7b&scope.project.id=e723438c25614a6babae92c8e882f147&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:26:10.400289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=6 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:10.548226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=8 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:22.102933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2a3bf-e0b4-46c9-82df-30600b4ac203 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:22.103484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2a3bf-e0b4-46c9-82df-30600b4ac203 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:22.103484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2a3bf-e0b4-46c9-82df-30600b4ac203 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:22.104839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 598/1201] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:22.110268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a24a-d01d-43a3-bd01-e364d0ddf5bc None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:22.110722 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a24a-d01d-43a3-bd01-e364d0ddf5bc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:22.110824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a24a-d01d-43a3-bd01-e364d0ddf5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:22.127270 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f929a24a-d01d-43a3-bd01-e364d0ddf5bc None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:22.128466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f929a24a-d01d-43a3-bd01-e364d0ddf5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:22.194872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 604/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:27:22.220775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2a3bf-e0b4-46c9-82df-30600b4ac203 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:22.322372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2a3bf-e0b4-46c9-82df-30600b4ac203 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:22.355862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8bde0-87be-47d7-b7ec-878127420aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:22.356573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8bde0-87be-47d7-b7ec-878127420aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:22.356856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8bde0-87be-47d7-b7ec-878127420aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:22.357131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8bde0-87be-47d7-b7ec-878127420aa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:22.367032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8bde0-87be-47d7-b7ec-878127420aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:22.368461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8bde0-87be-47d7-b7ec-878127420aa8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-9fd8bde0-87be-47d7-b7ec-878127420aa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:22.387922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:22.389104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:22.390771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:22.390771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:22.390771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:22.390771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:22.390771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 599/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:22.394129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929a24a-d01d-43a3-bd01-e364d0ddf5bc None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:22.497675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929a24a-d01d-43a3-bd01-e364d0ddf5bc None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:22.527370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:22.533344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38500d9-1f84-4907-b9bc-cad5dee194ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:22.534014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38500d9-1f84-4907-b9bc-cad5dee194ec None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:22.534294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38500d9-1f84-4907-b9bc-cad5dee194ec None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:22.534571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38500d9-1f84-4907-b9bc-cad5dee194ec None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:22.535299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38500d9-1f84-4907-b9bc-cad5dee194ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:22.535605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38500d9-1f84-4907-b9bc-cad5dee194ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:22.536973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38500d9-1f84-4907-b9bc-cad5dee194ec None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:22.548072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 605/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:27:22.553833 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8bde0-87be-47d7-b7ec-878127420aa8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:22.616396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:22.668578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8bde0-87be-47d7-b7ec-878127420aa8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:22.701307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c95a2-78ea-4f15-ac96-afdacb3eabe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:22.701970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c95a2-78ea-4f15-ac96-afdacb3eabe9 None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:22.702305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c95a2-78ea-4f15-ac96-afdacb3eabe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:22.703098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c95a2-78ea-4f15-ac96-afdacb3eabe9 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:22.703813 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c95a2-78ea-4f15-ac96-afdacb3eabe9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:22.705278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c95a2-78ea-4f15-ac96-afdacb3eabe9 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:22.878399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 600/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jul 14 11:27:22 2022] POST /identity/v3/users => generated 251 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:27:24.975986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529b92b-41a0-4129-9c15-1c294ea77688 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:24.976570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529b92b-41a0-4129-9c15-1c294ea77688 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:24.976570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529b92b-41a0-4129-9c15-1c294ea77688 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:24.977919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 606/1206] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:24.983026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238b52-5803-48fd-8447-6566df768430 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:24.983432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238b52-5803-48fd-8447-6566df768430 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:24.983615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238b52-5803-48fd-8447-6566df768430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:25.000126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-b5238b52-5803-48fd-8447-6566df768430 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:25.009263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b5238b52-5803-48fd-8447-6566df768430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:25.069434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 601/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:27:25.093964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529b92b-41a0-4129-9c15-1c294ea77688 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:25.192285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529b92b-41a0-4129-9c15-1c294ea77688 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:25.227903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e49462-e678-437c-b5d5-615cc6aae4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:25.228710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49462-e678-437c-b5d5-615cc6aae4bd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:25.228990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49462-e678-437c-b5d5-615cc6aae4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:25.229306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49462-e678-437c-b5d5-615cc6aae4bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:25.239924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e49462-e678-437c-b5d5-615cc6aae4bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b0e49462-e678-437c-b5d5-615cc6aae4bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:27:25.247829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:25.249024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:25.249969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 607/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:27:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:25.255373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5238b52-5803-48fd-8447-6566df768430 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:25.295274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:25.352914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5238b52-5803-48fd-8447-6566df768430 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:25.386305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338e57d-4cbe-437b-8a4b-84c1c3fdff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:25.386968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338e57d-4cbe-437b-8a4b-84c1c3fdff24 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:25.387248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338e57d-4cbe-437b-8a4b-84c1c3fdff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:25.387551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338e57d-4cbe-437b-8a4b-84c1c3fdff24 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:25.388295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338e57d-4cbe-437b-8a4b-84c1c3fdff24 None admin] RBAC: Adding query filter params (name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:25.388596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338e57d-4cbe-437b-8a4b-84c1c3fdff24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:25.390605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338e57d-4cbe-437b-8a4b-84c1c3fdff24 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:25.398776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 602/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:27:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:25.404572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e49462-e678-437c-b5d5-615cc6aae4bd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:25.455335 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:25.504549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e49462-e678-437c-b5d5-615cc6aae4bd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:25.537227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fa457-f35a-4314-a948-82596ab7a6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:25.538052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fa457-f35a-4314-a948-82596ab7a6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:25.538338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fa457-f35a-4314-a948-82596ab7a6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:25.538612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fa457-f35a-4314-a948-82596ab7a6a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:25.548956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fa457-f35a-4314-a948-82596ab7a6a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:25.551070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fa457-f35a-4314-a948-82596ab7a6a2 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-275fa457-f35a-4314-a948-82596ab7a6a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:25.568112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:25.569131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:25.570088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:25.570088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:25.570088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:25.570088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:25.570088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 608/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:25.575219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338e57d-4cbe-437b-8a4b-84c1c3fdff24 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:25.670147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:25.672971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338e57d-4cbe-437b-8a4b-84c1c3fdff24 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:25.712165 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd23a9-7d48-42af-8e2f-bbfb11b76889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:25.712824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd23a9-7d48-42af-8e2f-bbfb11b76889 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:25.713106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd23a9-7d48-42af-8e2f-bbfb11b76889 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:25.713409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd23a9-7d48-42af-8e2f-bbfb11b76889 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:25.714268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd23a9-7d48-42af-8e2f-bbfb11b76889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:25.714470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd23a9-7d48-42af-8e2f-bbfb11b76889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:25.715845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd23a9-7d48-42af-8e2f-bbfb11b76889 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:25.727019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 603/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:25.732852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fa457-f35a-4314-a948-82596ab7a6a2 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:25.830563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fa457-f35a-4314-a948-82596ab7a6a2 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:25.832066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:25.864085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12630eb0-7c53-46ef-b523-eee9816b2b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:25.864885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12630eb0-7c53-46ef-b523-eee9816b2b0f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:25.865199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12630eb0-7c53-46ef-b523-eee9816b2b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:25.865478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12630eb0-7c53-46ef-b523-eee9816b2b0f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:25.872727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12630eb0-7c53-46ef-b523-eee9816b2b0f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:25.874768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12630eb0-7c53-46ef-b523-eee9816b2b0f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-12630eb0-7c53-46ef-b523-eee9816b2b0f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:27:25.889225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:25.890524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:25.891868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 609/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:25.897591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd23a9-7d48-42af-8e2f-bbfb11b76889 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:25.998360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd23a9-7d48-42af-8e2f-bbfb11b76889 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:26.032032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224f5a3-5463-4085-82b0-1f4981578156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:26.032694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224f5a3-5463-4085-82b0-1f4981578156 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:26.032975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224f5a3-5463-4085-82b0-1f4981578156 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:26.033313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224f5a3-5463-4085-82b0-1f4981578156 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:26.034110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224f5a3-5463-4085-82b0-1f4981578156 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:26.034414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224f5a3-5463-4085-82b0-1f4981578156 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:26.035861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224f5a3-5463-4085-82b0-1f4981578156 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:26.053097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 604/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jul 14 11:27:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:26.059057 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12630eb0-7c53-46ef-b523-eee9816b2b0f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:26.159595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12630eb0-7c53-46ef-b523-eee9816b2b0f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:26.193116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46578d-b662-4982-96d5-dc92792c3714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:26.193786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46578d-b662-4982-96d5-dc92792c3714 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:26.194065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46578d-b662-4982-96d5-dc92792c3714 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:26.194332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46578d-b662-4982-96d5-dc92792c3714 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:26.224397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:26.261905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46578d-b662-4982-96d5-dc92792c3714 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:26.263395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46578d-b662-4982-96d5-dc92792c3714 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:26.360284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 610/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:27:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:27:26.366145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224f5a3-5463-4085-82b0-1f4981578156 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:26.385044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:26.466917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224f5a3-5463-4085-82b0-1f4981578156 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:26.501507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbffe95-f5da-47a6-a30c-0ee50b7940c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:26.502291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbffe95-f5da-47a6-a30c-0ee50b7940c8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:26.502573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbffe95-f5da-47a6-a30c-0ee50b7940c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:26.502849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbffe95-f5da-47a6-a30c-0ee50b7940c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:26.512849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbffe95-f5da-47a6-a30c-0ee50b7940c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:26.514340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbffe95-f5da-47a6-a30c-0ee50b7940c8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-2bbffe95-f5da-47a6-a30c-0ee50b7940c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:26.532971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:26.534099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:26.535021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:26.535021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:26.535021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:26.535021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:26.535021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 605/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:26.540209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46578d-b662-4982-96d5-dc92792c3714 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:26.599463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:26.637069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46578d-b662-4982-96d5-dc92792c3714 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:26.670175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b739e-22fd-4658-a9f2-13f4571d5968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:26.670856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b739e-22fd-4658-a9f2-13f4571d5968 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:26.671139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b739e-22fd-4658-a9f2-13f4571d5968 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:26.671415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b739e-22fd-4658-a9f2-13f4571d5968 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:26.672133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b739e-22fd-4658-a9f2-13f4571d5968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:26.672437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b739e-22fd-4658-a9f2-13f4571d5968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:26.673829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b739e-22fd-4658-a9f2-13f4571d5968 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:26.685258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 611/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:26.690921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbffe95-f5da-47a6-a30c-0ee50b7940c8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:26.758702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:26.791156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbffe95-f5da-47a6-a30c-0ee50b7940c8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:26.824776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec052146-4761-4559-9b9d-663ee1a41592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:26.825598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec052146-4761-4559-9b9d-663ee1a41592 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:26.825882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec052146-4761-4559-9b9d-663ee1a41592 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:26.826157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec052146-4761-4559-9b9d-663ee1a41592 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:26.833309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec052146-4761-4559-9b9d-663ee1a41592 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:26.834748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec052146-4761-4559-9b9d-663ee1a41592 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-ec052146-4761-4559-9b9d-663ee1a41592 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:27:26.849114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:26.850915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:26.852174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:27:26.852174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:27:26.852174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:27:26.852174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:27:26.852174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:27:26.852174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:27:26.852174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:26.852174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:26.852174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 606/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:27:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:27:26.856261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b739e-22fd-4658-a9f2-13f4571d5968 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:26.954625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b739e-22fd-4658-a9f2-13f4571d5968 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:26.976234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:26.987075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56740ad2-4f2f-459b-8c65-82eceb9939aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:26.987723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56740ad2-4f2f-459b-8c65-82eceb9939aa None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:26.988002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56740ad2-4f2f-459b-8c65-82eceb9939aa None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:26.988272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56740ad2-4f2f-459b-8c65-82eceb9939aa None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:26.989054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56740ad2-4f2f-459b-8c65-82eceb9939aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:26.989405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56740ad2-4f2f-459b-8c65-82eceb9939aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:26.990819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56740ad2-4f2f-459b-8c65-82eceb9939aa None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:27.002351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 612/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:27:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:27.008699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec052146-4761-4559-9b9d-663ee1a41592 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:27.109071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec052146-4761-4559-9b9d-663ee1a41592 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:27.136759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:27.143107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18e8a1-4475-46f9-90ed-cda388d3f021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:27.143762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e8a1-4475-46f9-90ed-cda388d3f021 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:27.144049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e8a1-4475-46f9-90ed-cda388d3f021 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:27.144329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e8a1-4475-46f9-90ed-cda388d3f021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:27.145172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18e8a1-4475-46f9-90ed-cda388d3f021 None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=fa879d581c6d44e18a8703e66ffbd549) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:27.145496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18e8a1-4475-46f9-90ed-cda388d3f021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:27.146937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18e8a1-4475-46f9-90ed-cda388d3f021 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:27.152309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 607/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:27:27 2022] GET /identity/v3/role_assignments?user.id=fa879d581c6d44e18a8703e66ffbd549&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:29.157715 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732b85f-435d-4ef4-8203-41780b242781 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:29.158237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732b85f-435d-4ef4-8203-41780b242781 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:29.158237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732b85f-435d-4ef4-8203-41780b242781 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:29.159557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 613/1220] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:29.164924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a908e9b-8b76-4205-8a2c-e9ba1e7fbbe6 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:29.167812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a908e9b-8b76-4205-8a2c-e9ba1e7fbbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:29.167812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a908e9b-8b76-4205-8a2c-e9ba1e7fbbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:29.182635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-3a908e9b-8b76-4205-8a2c-e9ba1e7fbbe6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:29.183870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3a908e9b-8b76-4205-8a2c-e9ba1e7fbbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:29.251761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 608/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:27:29.279309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732b85f-435d-4ef4-8203-41780b242781 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:29.381471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732b85f-435d-4ef4-8203-41780b242781 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:29.416811 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b47593-8adb-4d88-9643-fadc9cb83d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:29.417560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b47593-8adb-4d88-9643-fadc9cb83d75 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:29.417839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b47593-8adb-4d88-9643-fadc9cb83d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:29.418112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b47593-8adb-4d88-9643-fadc9cb83d75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:29.428724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b47593-8adb-4d88-9643-fadc9cb83d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-e7b47593-8adb-4d88-9643-fadc9cb83d75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:27:29.436486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:29.437643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:29.438842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 614/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:27:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:29.444276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a908e9b-8b76-4205-8a2c-e9ba1e7fbbe6 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:29.544509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a908e9b-8b76-4205-8a2c-e9ba1e7fbbe6 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:29.564567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:29.575114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fa9d1-f641-4718-b9f8-331a0a2ad6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:29.575782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fa9d1-f641-4718-b9f8-331a0a2ad6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:29.576072 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fa9d1-f641-4718-b9f8-331a0a2ad6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:29.576346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fa9d1-f641-4718-b9f8-331a0a2ad6f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:29.577661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fa9d1-f641-4718-b9f8-331a0a2ad6f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:29.577968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fa9d1-f641-4718-b9f8-331a0a2ad6f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:29.579336 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fa9d1-f641-4718-b9f8-331a0a2ad6f8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:29.587627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 609/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:27:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:29.594043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b47593-8adb-4d88-9643-fadc9cb83d75 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:29.636558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:29.698004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b47593-8adb-4d88-9643-fadc9cb83d75 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:29.731728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbb163-94aa-4ae5-a926-a34ba14ef199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:29.732528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb163-94aa-4ae5-a926-a34ba14ef199 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:29.732794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb163-94aa-4ae5-a926-a34ba14ef199 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:29.733069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb163-94aa-4ae5-a926-a34ba14ef199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:29.743797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbb163-94aa-4ae5-a926-a34ba14ef199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:29.745358 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbb163-94aa-4ae5-a926-a34ba14ef199 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-71cbb163-94aa-4ae5-a926-a34ba14ef199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:29.762379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:29.765589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:29.766875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:29.766875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:29.766875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:29.766875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:29.766875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 615/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:29.770233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fa9d1-f641-4718-b9f8-331a0a2ad6f8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:29.877488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fa9d1-f641-4718-b9f8-331a0a2ad6f8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:29.910413 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19bc15-6fd3-4c72-ab0e-162198e4f056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:29.911067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19bc15-6fd3-4c72-ab0e-162198e4f056 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:29.911348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19bc15-6fd3-4c72-ab0e-162198e4f056 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:29.911621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19bc15-6fd3-4c72-ab0e-162198e4f056 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:29.912386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19bc15-6fd3-4c72-ab0e-162198e4f056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:29.912691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19bc15-6fd3-4c72-ab0e-162198e4f056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:29.914086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19bc15-6fd3-4c72-ab0e-162198e4f056 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:29.925596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 610/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:29.931590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbb163-94aa-4ae5-a926-a34ba14ef199 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:30.030491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:30.037493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbb163-94aa-4ae5-a926-a34ba14ef199 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:30.070414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c0930-dd07-4495-9d2e-7325e474cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:30.071211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c0930-dd07-4495-9d2e-7325e474cb6f None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:30.071490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c0930-dd07-4495-9d2e-7325e474cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:30.071761 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c0930-dd07-4495-9d2e-7325e474cb6f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:30.079272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c0930-dd07-4495-9d2e-7325e474cb6f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:30.080693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c0930-dd07-4495-9d2e-7325e474cb6f None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-5a5c0930-dd07-4495-9d2e-7325e474cb6f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:27:30.094882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:30.096005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:30.096980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 616/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:30.102224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19bc15-6fd3-4c72-ab0e-162198e4f056 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:30.148972 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:30.201205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19bc15-6fd3-4c72-ab0e-162198e4f056 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:30.234799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a848056-133f-47fb-af10-6fbd45451e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:30.237815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a848056-133f-47fb-af10-6fbd45451e9b None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:30.237815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a848056-133f-47fb-af10-6fbd45451e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:30.239593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a848056-133f-47fb-af10-6fbd45451e9b None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:30.240476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a848056-133f-47fb-af10-6fbd45451e9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:30.240793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a848056-133f-47fb-af10-6fbd45451e9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:30.243118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a848056-133f-47fb-af10-6fbd45451e9b None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:30.263156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 611/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jul 14 11:27:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:30.270787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c0930-dd07-4495-9d2e-7325e474cb6f None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:30.367569 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c0930-dd07-4495-9d2e-7325e474cb6f None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:30.400763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f27b5-b3bc-45f9-b33d-80e8aca6decd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:30.401605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f27b5-b3bc-45f9-b33d-80e8aca6decd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:30.401852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f27b5-b3bc-45f9-b33d-80e8aca6decd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:30.402123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f27b5-b3bc-45f9-b33d-80e8aca6decd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:30.412287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f27b5-b3bc-45f9-b33d-80e8aca6decd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:30.413765 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f27b5-b3bc-45f9-b33d-80e8aca6decd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-a89f27b5-b3bc-45f9-b33d-80e8aca6decd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:30.431315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:30.436230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:30.437998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:30.437998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:30.437998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:30.437998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:30.437998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 617/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:30.439119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a848056-133f-47fb-af10-6fbd45451e9b None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:30.539958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a848056-133f-47fb-af10-6fbd45451e9b None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:30.542421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:30.576849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1d06a-ae02-4f13-be68-d79a5bfebb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:30.577573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1d06a-ae02-4f13-be68-d79a5bfebb65 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:30.577859 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1d06a-ae02-4f13-be68-d79a5bfebb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:30.578137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1d06a-ae02-4f13-be68-d79a5bfebb65 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:30.578877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1d06a-ae02-4f13-be68-d79a5bfebb65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:30.579185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1d06a-ae02-4f13-be68-d79a5bfebb65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:30.580560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1d06a-ae02-4f13-be68-d79a5bfebb65 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:30.592254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 612/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:30.598443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f27b5-b3bc-45f9-b33d-80e8aca6decd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:30.607716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:30.697809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f27b5-b3bc-45f9-b33d-80e8aca6decd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:30.731997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c68ee8-1f0d-49ea-a31d-932ef1ebd24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:30.732777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c68ee8-1f0d-49ea-a31d-932ef1ebd24e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:30.733060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c68ee8-1f0d-49ea-a31d-932ef1ebd24e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:30.733900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c68ee8-1f0d-49ea-a31d-932ef1ebd24e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:30.740391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c68ee8-1f0d-49ea-a31d-932ef1ebd24e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:30.741857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c68ee8-1f0d-49ea-a31d-932ef1ebd24e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-67c68ee8-1f0d-49ea-a31d-932ef1ebd24e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:27:30.755094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:30.756154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:30.757373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:27:30.757373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:27:30.757373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:27:30.757373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:27:30.757373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:27:30.757373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:27:30.757373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:30.757373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:30.757373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 618/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:27:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:30.762064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1d06a-ae02-4f13-be68-d79a5bfebb65 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:30.862005 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1d06a-ae02-4f13-be68-d79a5bfebb65 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:30.894795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dc1e4-9ce0-4a6d-94c7-d722300b1559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:30.895471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dc1e4-9ce0-4a6d-94c7-d722300b1559 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:30.895738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dc1e4-9ce0-4a6d-94c7-d722300b1559 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:30.896019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dc1e4-9ce0-4a6d-94c7-d722300b1559 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:30.896819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35dc1e4-9ce0-4a6d-94c7-d722300b1559 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:30.897126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35dc1e4-9ce0-4a6d-94c7-d722300b1559 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:30.898629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35dc1e4-9ce0-4a6d-94c7-d722300b1559 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:30.912203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 613/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:27:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:30.918506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c68ee8-1f0d-49ea-a31d-932ef1ebd24e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:30.984340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:31.017854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c68ee8-1f0d-49ea-a31d-932ef1ebd24e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:31.050721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e77011-46cf-43f3-bdc7-55f85d580d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:31.051600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e77011-46cf-43f3-bdc7-55f85d580d30 None admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:31.051911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e77011-46cf-43f3-bdc7-55f85d580d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:31.052153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e77011-46cf-43f3-bdc7-55f85d580d30 None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/fa879d581c6d44e18a8703e66ffbd549/roles/60130d2ba383400091dc5f253d40a43a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:31.067841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e77011-46cf-43f3-bdc7-55f85d580d30 None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=fa879d581c6d44e18a8703e66ffbd549, role_id=60130d2ba383400091dc5f253d40a43a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:31.069331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e77011-46cf-43f3-bdc7-55f85d580d30 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:31.172145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 619/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:27:30 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/fa879d581c6d44e18a8703e66ffbd549/roles/60130d2ba383400091dc5f253d40a43a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:27:33.210712 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865789-2d61-40e5-806c-b0fc42fd683c None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:33.211231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865789-2d61-40e5-806c-b0fc42fd683c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:33.211231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865789-2d61-40e5-806c-b0fc42fd683c None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:33.212609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 614/1233] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:33.218481 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197251-8020-4d35-82d3-9e2d2aa43865 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:33.218913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197251-8020-4d35-82d3-9e2d2aa43865 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:33.219034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197251-8020-4d35-82d3-9e2d2aa43865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:33.235762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-17197251-8020-4d35-82d3-9e2d2aa43865 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:33.236950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-17197251-8020-4d35-82d3-9e2d2aa43865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:33.302032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 620/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 11:27:33.326011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd865789-2d61-40e5-806c-b0fc42fd683c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:33.352050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:33.428298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd865789-2d61-40e5-806c-b0fc42fd683c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:33.462699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa7158-05a6-4413-bdf9-97bc0e902652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:33.463438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa7158-05a6-4413-bdf9-97bc0e902652 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:33.463718 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa7158-05a6-4413-bdf9-97bc0e902652 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:33.464027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa7158-05a6-4413-bdf9-97bc0e902652 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:33.474753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa7158-05a6-4413-bdf9-97bc0e902652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-96fa7158-05a6-4413-bdf9-97bc0e902652 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:27:33.482467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:33.483632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:33.484542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 615/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jul 14 11:27:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:33.489796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197251-8020-4d35-82d3-9e2d2aa43865 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:33.537710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:33.591824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197251-8020-4d35-82d3-9e2d2aa43865 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:33.626076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3a533-a3f2-4209-812d-39b65a370363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:33.626749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3a533-a3f2-4209-812d-39b65a370363 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:33.627032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3a533-a3f2-4209-812d-39b65a370363 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:33.627306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3a533-a3f2-4209-812d-39b65a370363 None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:33.628057 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3a533-a3f2-4209-812d-39b65a370363 None admin] RBAC: Adding query filter params (name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:33.628380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3a533-a3f2-4209-812d-39b65a370363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:33.629770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3a533-a3f2-4209-812d-39b65a370363 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:33.638128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 621/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jul 14 11:27:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:33.644839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa7158-05a6-4413-bdf9-97bc0e902652 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:33.729714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:33.744768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa7158-05a6-4413-bdf9-97bc0e902652 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:33.778353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb4053-ff20-46f4-9786-6cc3efa7d36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:33.779141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb4053-ff20-46f4-9786-6cc3efa7d36a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:33.779426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb4053-ff20-46f4-9786-6cc3efa7d36a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:33.779701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb4053-ff20-46f4-9786-6cc3efa7d36a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:33.789728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb4053-ff20-46f4-9786-6cc3efa7d36a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:33.791176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb4053-ff20-46f4-9786-6cc3efa7d36a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-6ffb4053-ff20-46f4-9786-6cc3efa7d36a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:33.809565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:33.810695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:33.812196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:33.812196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:33.812196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:33.812196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:33.812196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 616/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:27:33.817447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3a533-a3f2-4209-812d-39b65a370363 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:33.918635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3a533-a3f2-4209-812d-39b65a370363 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:33.927841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:33.953447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbaa65-f675-4cb9-b1fb-253207e97f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:33.956103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbaa65-f675-4cb9-b1fb-253207e97f98 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:33.956103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbaa65-f675-4cb9-b1fb-253207e97f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:33.956103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbaa65-f675-4cb9-b1fb-253207e97f98 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:33.956103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbaa65-f675-4cb9-b1fb-253207e97f98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:33.956103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbaa65-f675-4cb9-b1fb-253207e97f98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:33.957102 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbaa65-f675-4cb9-b1fb-253207e97f98 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:33.968741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 622/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:33.974687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb4053-ff20-46f4-9786-6cc3efa7d36a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:34.073634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb4053-ff20-46f4-9786-6cc3efa7d36a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:34.106163 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b35a6-9789-422f-a19f-c43c58634507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:34.106952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b35a6-9789-422f-a19f-c43c58634507 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:34.107235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b35a6-9789-422f-a19f-c43c58634507 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:34.107509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b35a6-9789-422f-a19f-c43c58634507 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:34.110733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:34.115291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b35a6-9789-422f-a19f-c43c58634507 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:34.116737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b35a6-9789-422f-a19f-c43c58634507 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-c69b35a6-9789-422f-a19f-c43c58634507 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:27:34.131587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:34.133381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:34.135663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 617/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:27:34.140667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbaa65-f675-4cb9-b1fb-253207e97f98 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:34.237717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbaa65-f675-4cb9-b1fb-253207e97f98 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:34.272273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6991b0-d4e6-45d3-97d3-7434c8259415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:34.272937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6991b0-d4e6-45d3-97d3-7434c8259415 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:34.273247 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6991b0-d4e6-45d3-97d3-7434c8259415 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:34.273525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6991b0-d4e6-45d3-97d3-7434c8259415 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:34.274360 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6991b0-d4e6-45d3-97d3-7434c8259415 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:34.274671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6991b0-d4e6-45d3-97d3-7434c8259415 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:34.276098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6991b0-d4e6-45d3-97d3-7434c8259415 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:34.292891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 623/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jul 14 11:27:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:34.299265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b35a6-9789-422f-a19f-c43c58634507 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:34.398483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b35a6-9789-422f-a19f-c43c58634507 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:34.433650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5760a-08c5-4d84-b11d-563b0568c0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:34.434297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5760a-08c5-4d84-b11d-563b0568c0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:34.434581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5760a-08c5-4d84-b11d-563b0568c0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:34.435064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5760a-08c5-4d84-b11d-563b0568c0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:34.496352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:34.502789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5760a-08c5-4d84-b11d-563b0568c0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:34.504286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5760a-08c5-4d84-b11d-563b0568c0f4 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:34.598912 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 618/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:27:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:27:34.604683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6991b0-d4e6-45d3-97d3-7434c8259415 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:34.631389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:34.699608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6991b0-d4e6-45d3-97d3-7434c8259415 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:34.732109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be8c5a-e400-492d-916a-5df4d056b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:34.732900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be8c5a-e400-492d-916a-5df4d056b459 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:34.733201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be8c5a-e400-492d-916a-5df4d056b459 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:34.733479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be8c5a-e400-492d-916a-5df4d056b459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:34.743498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be8c5a-e400-492d-916a-5df4d056b459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:34.744928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be8c5a-e400-492d-916a-5df4d056b459 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-f1be8c5a-e400-492d-916a-5df4d056b459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:34.761882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:34.762959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:34.764110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:34.764110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:34.764110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:34.764110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:34.764110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 624/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:34.769654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5760a-08c5-4d84-b11d-563b0568c0f4 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:34.863585 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:34.870137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5760a-08c5-4d84-b11d-563b0568c0f4 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:34.903943 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad643c2-b5ce-430c-921c-b3edcd9bf942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:34.904610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad643c2-b5ce-430c-921c-b3edcd9bf942 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:34.905527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad643c2-b5ce-430c-921c-b3edcd9bf942 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:34.905811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad643c2-b5ce-430c-921c-b3edcd9bf942 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:34.906540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad643c2-b5ce-430c-921c-b3edcd9bf942 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:34.906849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad643c2-b5ce-430c-921c-b3edcd9bf942 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:34.908277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad643c2-b5ce-430c-921c-b3edcd9bf942 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:34.919479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 619/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:34.925303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be8c5a-e400-492d-916a-5df4d056b459 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:35.008640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:35.030581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be8c5a-e400-492d-916a-5df4d056b459 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:35.064120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bd369-e78e-4459-9bdf-12a109678cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:35.064909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bd369-e78e-4459-9bdf-12a109678cef None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:35.065249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bd369-e78e-4459-9bdf-12a109678cef None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:35.065525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bd369-e78e-4459-9bdf-12a109678cef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:35.072391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bd369-e78e-4459-9bdf-12a109678cef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:35.076186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bd369-e78e-4459-9bdf-12a109678cef None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-420bd369-e78e-4459-9bdf-12a109678cef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:27:35.087845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:35.088887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:35.089821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:27:35.089821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:27:35.089821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:27:35.089821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:27:35.089821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:27:35.089821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:27:35.089821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:35.089821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:35.089821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 625/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:27:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:35.096054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad643c2-b5ce-430c-921c-b3edcd9bf942 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:35.194314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad643c2-b5ce-430c-921c-b3edcd9bf942 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:35.243974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba513fc0-20cc-4bdc-b951-886a2634c586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:35.244706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba513fc0-20cc-4bdc-b951-886a2634c586 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:35.244955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba513fc0-20cc-4bdc-b951-886a2634c586 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:35.245325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba513fc0-20cc-4bdc-b951-886a2634c586 None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:35.246118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba513fc0-20cc-4bdc-b951-886a2634c586 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:35.246422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba513fc0-20cc-4bdc-b951-886a2634c586 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:35.247848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba513fc0-20cc-4bdc-b951-886a2634c586 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:35.267690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 620/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:27:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:35.272677 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bd369-e78e-4459-9bdf-12a109678cef None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:35.375745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bd369-e78e-4459-9bdf-12a109678cef None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:35.410053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad0acf-ba80-47cd-931d-711adab0c4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:35.410714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad0acf-ba80-47cd-931d-711adab0c4de None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:35.410989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad0acf-ba80-47cd-931d-711adab0c4de None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:35.411264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad0acf-ba80-47cd-931d-711adab0c4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:35.412114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad0acf-ba80-47cd-931d-711adab0c4de None admin] RBAC: Adding query filter params (role.id=60130d2ba383400091dc5f253d40a43a, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=fa879d581c6d44e18a8703e66ffbd549) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:35.412420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad0acf-ba80-47cd-931d-711adab0c4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:35.413873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad0acf-ba80-47cd-931d-711adab0c4de None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:35.419718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 626/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:27:35 2022] GET /identity/v3/role_assignments?user.id=fa879d581c6d44e18a8703e66ffbd549&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=60130d2ba383400091dc5f253d40a43a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:37.508070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba95377-1de0-472d-9b9d-51c74d311399 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:37.508674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba95377-1de0-472d-9b9d-51c74d311399 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:37.508674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba95377-1de0-472d-9b9d-51c74d311399 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:37.510024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 621/1247] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:37.515227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f89bb6-8236-4072-8b74-26e49227a7e9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:37.515694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f89bb6-8236-4072-8b74-26e49227a7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:37.515805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f89bb6-8236-4072-8b74-26e49227a7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:37.520814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:37.533499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-26f89bb6-8236-4072-8b74-26e49227a7e9 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:37.534692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-26f89bb6-8236-4072-8b74-26e49227a7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:37.598884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 627/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:27:37.622617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba95377-1de0-472d-9b9d-51c74d311399 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:37.643080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:37.724979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba95377-1de0-472d-9b9d-51c74d311399 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:37.757650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8fbee-fd46-4693-8985-82ad99dfb298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:37.758477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8fbee-fd46-4693-8985-82ad99dfb298 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:37.758708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8fbee-fd46-4693-8985-82ad99dfb298 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:37.758983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8fbee-fd46-4693-8985-82ad99dfb298 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:37.769351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8fbee-fd46-4693-8985-82ad99dfb298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-4fd8fbee-fd46-4693-8985-82ad99dfb298 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:27:37.776992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:37.784118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:37.784985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 622/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:27:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:37.785731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f89bb6-8236-4072-8b74-26e49227a7e9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:37.887086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f89bb6-8236-4072-8b74-26e49227a7e9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:37.917428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:37.922331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9279d40-9201-403e-8e99-ad6e1e4a6d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:37.922973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9279d40-9201-403e-8e99-ad6e1e4a6d2e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:37.923253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9279d40-9201-403e-8e99-ad6e1e4a6d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:37.923524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9279d40-9201-403e-8e99-ad6e1e4a6d2e None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:37.924269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9279d40-9201-403e-8e99-ad6e1e4a6d2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:37.924564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9279d40-9201-403e-8e99-ad6e1e4a6d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:37.926002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9279d40-9201-403e-8e99-ad6e1e4a6d2e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:37.933986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 628/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:27:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:37.939793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8fbee-fd46-4693-8985-82ad99dfb298 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:38.017589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:38.040230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8fbee-fd46-4693-8985-82ad99dfb298 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:38.073660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda058b-db17-443f-bbff-a41962d7e7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:38.074466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda058b-db17-443f-bbff-a41962d7e7db None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:38.074751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda058b-db17-443f-bbff-a41962d7e7db None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:38.075027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda058b-db17-443f-bbff-a41962d7e7db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:38.086437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda058b-db17-443f-bbff-a41962d7e7db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:38.088983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda058b-db17-443f-bbff-a41962d7e7db None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-3dda058b-db17-443f-bbff-a41962d7e7db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:38.105551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:38.106629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:38.107694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:38.107694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:38.107694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:38.107694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:38.107694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 623/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:38.112507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9279d40-9201-403e-8e99-ad6e1e4a6d2e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:38.212160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9279d40-9201-403e-8e99-ad6e1e4a6d2e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:38.245224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cb72b-84d1-443e-840e-a7f7257fb3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:38.245889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb72b-84d1-443e-840e-a7f7257fb3cb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:38.246176 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb72b-84d1-443e-840e-a7f7257fb3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:38.246444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb72b-84d1-443e-840e-a7f7257fb3cb None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:38.247162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cb72b-84d1-443e-840e-a7f7257fb3cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:38.247466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cb72b-84d1-443e-840e-a7f7257fb3cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:38.248825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cb72b-84d1-443e-840e-a7f7257fb3cb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:38.260644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 629/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:38.266263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda058b-db17-443f-bbff-a41962d7e7db None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:38.367036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda058b-db17-443f-bbff-a41962d7e7db None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:38.392755 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:38.400474 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d15df-a63d-463a-b01a-196dba01c357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:38.401335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d15df-a63d-463a-b01a-196dba01c357 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:38.401621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d15df-a63d-463a-b01a-196dba01c357 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:38.401898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d15df-a63d-463a-b01a-196dba01c357 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:38.410101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d15df-a63d-463a-b01a-196dba01c357 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:38.411651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d15df-a63d-463a-b01a-196dba01c357 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-ea8d15df-a63d-463a-b01a-196dba01c357 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:27:38.428629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:38.429731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:38.430680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 624/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:38.435688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cb72b-84d1-443e-840e-a7f7257fb3cb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:38.491704 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:38.535848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cb72b-84d1-443e-840e-a7f7257fb3cb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:38.570276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7158a44-c851-4f98-986d-4e227825ecc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:38.570945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158a44-c851-4f98-986d-4e227825ecc9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:38.571234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158a44-c851-4f98-986d-4e227825ecc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:38.571505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158a44-c851-4f98-986d-4e227825ecc9 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:38.572386 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158a44-c851-4f98-986d-4e227825ecc9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:38.572696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158a44-c851-4f98-986d-4e227825ecc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:38.574153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158a44-c851-4f98-986d-4e227825ecc9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:38.591670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 630/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jul 14 11:27:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:38.598002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d15df-a63d-463a-b01a-196dba01c357 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:38.699533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d15df-a63d-463a-b01a-196dba01c357 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:38.734759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc2a28-d96e-4cda-b716-3423c6acb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:38.735410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc2a28-d96e-4cda-b716-3423c6acb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:38.735692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc2a28-d96e-4cda-b716-3423c6acb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:38.735965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc2a28-d96e-4cda-b716-3423c6acb2b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:38.791831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:38.804366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc2a28-d96e-4cda-b716-3423c6acb2b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:38.805857 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc2a28-d96e-4cda-b716-3423c6acb2b7 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:38.904049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 625/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:27:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:27:38.910530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7158a44-c851-4f98-986d-4e227825ecc9 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:39.009795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7158a44-c851-4f98-986d-4e227825ecc9 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:39.023154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:39.043699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8c9cc-30c2-4a10-9074-ee81399e14a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:39.044492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8c9cc-30c2-4a10-9074-ee81399e14a8 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:39.044773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8c9cc-30c2-4a10-9074-ee81399e14a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:39.045048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8c9cc-30c2-4a10-9074-ee81399e14a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:39.055582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8c9cc-30c2-4a10-9074-ee81399e14a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:39.057020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8c9cc-30c2-4a10-9074-ee81399e14a8 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-deb8c9cc-30c2-4a10-9074-ee81399e14a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:39.074730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:39.075812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:39.076975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:39.076975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:39.076975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:39.076975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:39.076975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 631/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:39.081926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc2a28-d96e-4cda-b716-3423c6acb2b7 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:39.169251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:39.181579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc2a28-d96e-4cda-b716-3423c6acb2b7 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:39.215906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0e4c9-9673-4a3e-a74a-f1bab66b6ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:39.216562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0e4c9-9673-4a3e-a74a-f1bab66b6ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:39.216851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0e4c9-9673-4a3e-a74a-f1bab66b6ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:39.217700 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0e4c9-9673-4a3e-a74a-f1bab66b6ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:39.218442 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0e4c9-9673-4a3e-a74a-f1bab66b6ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:39.218746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0e4c9-9673-4a3e-a74a-f1bab66b6ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:39.220123 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0e4c9-9673-4a3e-a74a-f1bab66b6ad8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:39.231514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 626/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:39.237684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8c9cc-30c2-4a10-9074-ee81399e14a8 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:39.337998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8c9cc-30c2-4a10-9074-ee81399e14a8 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:39.391989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b2828-307b-4e6f-8285-43976c43464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:39.392793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b2828-307b-4e6f-8285-43976c43464e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:39.393069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b2828-307b-4e6f-8285-43976c43464e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:39.393365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b2828-307b-4e6f-8285-43976c43464e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:39.401384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b2828-307b-4e6f-8285-43976c43464e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:39.402847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b2828-307b-4e6f-8285-43976c43464e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-eb9b2828-307b-4e6f-8285-43976c43464e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:27:39.419342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:39.420514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:39.421850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:27:39.421850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:27:39.421850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:27:39.421850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:27:39.421850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:27:39.421850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:27:39.421850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:39.421850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:39.421850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 632/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:27:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:39.426919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0e4c9-9673-4a3e-a74a-f1bab66b6ad8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:39.532061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0e4c9-9673-4a3e-a74a-f1bab66b6ad8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:39.566344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaadc3-89a2-4aef-8955-af821c21894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:39.567250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaadc3-89a2-4aef-8955-af821c21894f None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:39.567530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaadc3-89a2-4aef-8955-af821c21894f None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:39.567803 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaadc3-89a2-4aef-8955-af821c21894f None admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:39.568588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaadc3-89a2-4aef-8955-af821c21894f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:39.568891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaadc3-89a2-4aef-8955-af821c21894f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:39.570350 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaadc3-89a2-4aef-8955-af821c21894f None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:39.582593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 627/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:27:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:39.588698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b2828-307b-4e6f-8285-43976c43464e None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:39.614818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:39.697046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b2828-307b-4e6f-8285-43976c43464e None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:39.736341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c6a41-fccc-4a6d-bdff-e64ac4ce086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:39.737003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c6a41-fccc-4a6d-bdff-e64ac4ce086e None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:39.737321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c6a41-fccc-4a6d-bdff-e64ac4ce086e None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:39.737596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c6a41-fccc-4a6d-bdff-e64ac4ce086e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:39.741278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c6a41-fccc-4a6d-bdff-e64ac4ce086e None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=fa879d581c6d44e18a8703e66ffbd549) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:39.741278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c6a41-fccc-4a6d-bdff-e64ac4ce086e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:39.741278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c6a41-fccc-4a6d-bdff-e64ac4ce086e None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:39.746320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 633/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:27:39 2022] GET /identity/v3/role_assignments?user.id=fa879d581c6d44e18a8703e66ffbd549&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:41.809818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a193c-a1b5-423a-80db-d79f7c4553fe None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:41.810233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a193c-a1b5-423a-80db-d79f7c4553fe None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:41.810382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a193c-a1b5-423a-80db-d79f7c4553fe None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:41.811798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 628/1261] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:41.817100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a8703-f8e4-4207-b9a6-899cbbd88dbd None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:41.817539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a8703-f8e4-4207-b9a6-899cbbd88dbd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:41.817673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a8703-f8e4-4207-b9a6-899cbbd88dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:41.835242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cd8a8703-f8e4-4207-b9a6-899cbbd88dbd None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:41.836426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cd8a8703-f8e4-4207-b9a6-899cbbd88dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:41.904108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 634/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:27:41.927033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a193c-a1b5-423a-80db-d79f7c4553fe None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:41.938402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:42.026736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a193c-a1b5-423a-80db-d79f7c4553fe None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:42.060191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a7090-42a9-4fe5-86b7-6a13ee025c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:42.061473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a7090-42a9-4fe5-86b7-6a13ee025c43 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:42.061749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a7090-42a9-4fe5-86b7-6a13ee025c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:42.062016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a7090-42a9-4fe5-86b7-6a13ee025c43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:42.072322 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a7090-42a9-4fe5-86b7-6a13ee025c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-726a7090-42a9-4fe5-86b7-6a13ee025c43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:27:42.080376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:42.083191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:42.084075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 629/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:27:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:27:42.087665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a8703-f8e4-4207-b9a6-899cbbd88dbd None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:42.106111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:42.184659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a8703-f8e4-4207-b9a6-899cbbd88dbd None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:42.219838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbaa01-25e4-4ef5-874e-bf013ab934dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:42.220501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbaa01-25e4-4ef5-874e-bf013ab934dc None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:42.220783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbaa01-25e4-4ef5-874e-bf013ab934dc None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:42.221054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbaa01-25e4-4ef5-874e-bf013ab934dc None admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:42.221956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbaa01-25e4-4ef5-874e-bf013ab934dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:42.222305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbaa01-25e4-4ef5-874e-bf013ab934dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:42.223663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbaa01-25e4-4ef5-874e-bf013ab934dc None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:42.231595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 635/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:27:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:42.237425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a7090-42a9-4fe5-86b7-6a13ee025c43 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:42.302698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:42.335622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a7090-42a9-4fe5-86b7-6a13ee025c43 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:42.368561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66304db1-615b-4418-a7eb-afaee4620388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:42.369371 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66304db1-615b-4418-a7eb-afaee4620388 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:42.369661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66304db1-615b-4418-a7eb-afaee4620388 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:42.369936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66304db1-615b-4418-a7eb-afaee4620388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:42.381637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66304db1-615b-4418-a7eb-afaee4620388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:42.383121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66304db1-615b-4418-a7eb-afaee4620388 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-66304db1-615b-4418-a7eb-afaee4620388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:42.399885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:42.401632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:42.403484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:42.403484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:42.403484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:42.403484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:42.403484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 630/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:42.408397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbaa01-25e4-4ef5-874e-bf013ab934dc None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:42.490461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:42.508655 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbaa01-25e4-4ef5-874e-bf013ab934dc None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:42.541129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cfc87-c2bc-480b-bca3-2f8036d23108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:42.541824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cfc87-c2bc-480b-bca3-2f8036d23108 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:42.542109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cfc87-c2bc-480b-bca3-2f8036d23108 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:42.542392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cfc87-c2bc-480b-bca3-2f8036d23108 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:42.543112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cfc87-c2bc-480b-bca3-2f8036d23108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:42.543418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cfc87-c2bc-480b-bca3-2f8036d23108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:42.544781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cfc87-c2bc-480b-bca3-2f8036d23108 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:42.556258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 636/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:42.563207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66304db1-615b-4418-a7eb-afaee4620388 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:42.663044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66304db1-615b-4418-a7eb-afaee4620388 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:42.682549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:42.696064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee4ce3-0e0c-420d-b051-85f676ba3881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:42.696858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee4ce3-0e0c-420d-b051-85f676ba3881 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:42.697137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee4ce3-0e0c-420d-b051-85f676ba3881 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:42.697433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee4ce3-0e0c-420d-b051-85f676ba3881 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:42.704911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee4ce3-0e0c-420d-b051-85f676ba3881 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:42.706382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee4ce3-0e0c-420d-b051-85f676ba3881 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-2dee4ce3-0e0c-420d-b051-85f676ba3881 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:27:42.720809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:42.722010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:42.723203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 631/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:42.728148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cfc87-c2bc-480b-bca3-2f8036d23108 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:42.826115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cfc87-c2bc-480b-bca3-2f8036d23108 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:42.859111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618c655-ef66-4fae-a21c-ddb422b3db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:42.859768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618c655-ef66-4fae-a21c-ddb422b3db41 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:42.860046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618c655-ef66-4fae-a21c-ddb422b3db41 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:42.860318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618c655-ef66-4fae-a21c-ddb422b3db41 None admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:42.861203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618c655-ef66-4fae-a21c-ddb422b3db41 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:42.861509 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618c655-ef66-4fae-a21c-ddb422b3db41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:42.862933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618c655-ef66-4fae-a21c-ddb422b3db41 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:42.872000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:42.880784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 637/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jul 14 11:27:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:42.886552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee4ce3-0e0c-420d-b051-85f676ba3881 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:42.986211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee4ce3-0e0c-420d-b051-85f676ba3881 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:43.023473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93479e0-2af0-4463-ab78-a47b2d22841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:43.024314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93479e0-2af0-4463-ab78-a47b2d22841d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:43.024602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93479e0-2af0-4463-ab78-a47b2d22841d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:43.024880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93479e0-2af0-4463-ab78-a47b2d22841d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:43.035497 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93479e0-2af0-4463-ab78-a47b2d22841d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:43.036940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93479e0-2af0-4463-ab78-a47b2d22841d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-e93479e0-2af0-4463-ab78-a47b2d22841d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:43.054181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:43.055264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:43.061611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:43.061611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:43.061611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:43.061611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:43.061611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 632/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:43.061973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618c655-ef66-4fae-a21c-ddb422b3db41 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:43.161628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618c655-ef66-4fae-a21c-ddb422b3db41 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:43.197318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb613d-f44f-4bba-b579-8b03d212f06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:43.198016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb613d-f44f-4bba-b579-8b03d212f06d None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:43.198345 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb613d-f44f-4bba-b579-8b03d212f06d None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:43.198619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb613d-f44f-4bba-b579-8b03d212f06d None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:43.199378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb613d-f44f-4bba-b579-8b03d212f06d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:43.199685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb613d-f44f-4bba-b579-8b03d212f06d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:43.201044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb613d-f44f-4bba-b579-8b03d212f06d None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:43.212690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 638/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:27:43.218335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93479e0-2af0-4463-ab78-a47b2d22841d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:43.250550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:43.317932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93479e0-2af0-4463-ab78-a47b2d22841d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:43.350894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407bf9d-55c4-41b7-b1b1-546e133144f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:43.351673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407bf9d-55c4-41b7-b1b1-546e133144f8 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:43.351961 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407bf9d-55c4-41b7-b1b1-546e133144f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:43.352241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407bf9d-55c4-41b7-b1b1-546e133144f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:43.358909 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407bf9d-55c4-41b7-b1b1-546e133144f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:43.360355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407bf9d-55c4-41b7-b1b1-546e133144f8 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-8407bf9d-55c4-41b7-b1b1-546e133144f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:27:43.375775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:43.377007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:43.377975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:27:43.377975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:27:43.377975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:27:43.377975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:27:43.377975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:27:43.377975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:27:43.377975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:43.377975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:43.377975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 633/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:27:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:43.383168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb613d-f44f-4bba-b579-8b03d212f06d None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:43.452593 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:43.492088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb613d-f44f-4bba-b579-8b03d212f06d None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:43.525488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164dba54-f918-40ea-a4ec-2ea99afb9316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:43.526150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dba54-f918-40ea-a4ec-2ea99afb9316 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:43.526434 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dba54-f918-40ea-a4ec-2ea99afb9316 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:43.526708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dba54-f918-40ea-a4ec-2ea99afb9316 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:43.527488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164dba54-f918-40ea-a4ec-2ea99afb9316 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:43.527796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164dba54-f918-40ea-a4ec-2ea99afb9316 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:43.529259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164dba54-f918-40ea-a4ec-2ea99afb9316 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:43.541912 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 639/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:27:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:43.547925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407bf9d-55c4-41b7-b1b1-546e133144f8 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:43.640200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:43.649121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407bf9d-55c4-41b7-b1b1-546e133144f8 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:43.682226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e6326-55cd-408f-95bb-fcf80dfeaa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:43.683098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e6326-55cd-408f-95bb-fcf80dfeaa5d None admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:43.683383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e6326-55cd-408f-95bb-fcf80dfeaa5d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:43.683657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e6326-55cd-408f-95bb-fcf80dfeaa5d None admin] PATH_INFO: `/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/fa879d581c6d44e18a8703e66ffbd549/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:43.686165 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e6326-55cd-408f-95bb-fcf80dfeaa5d None admin] RBAC: Authorizing `identity:create_grant(project_id=69cec1c5d7884e1b94e44e7b586ef19a, user_id=fa879d581c6d44e18a8703e66ffbd549, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:43.687608 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e6326-55cd-408f-95bb-fcf80dfeaa5d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:43.788852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 634/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jul 14 11:27:43 2022] PUT /identity/v3/projects/69cec1c5d7884e1b94e44e7b586ef19a/users/fa879d581c6d44e18a8703e66ffbd549/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:27:45.797683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45cafd-b656-45de-8d53-8bde10ca71b3 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:45.798213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45cafd-b656-45de-8d53-8bde10ca71b3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:45.798213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45cafd-b656-45de-8d53-8bde10ca71b3 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:45.799588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 640/1274] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:45.804796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0711740-7672-4cf8-8feb-905c0b7fbd14 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:45.805187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0711740-7672-4cf8-8feb-905c0b7fbd14 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:45.805483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0711740-7672-4cf8-8feb-905c0b7fbd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:45.822568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c0711740-7672-4cf8-8feb-905c0b7fbd14 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:45.823763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c0711740-7672-4cf8-8feb-905c0b7fbd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:45.893655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 635/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:27:45.917667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45cafd-b656-45de-8d53-8bde10ca71b3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:46.018439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45cafd-b656-45de-8d53-8bde10ca71b3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:46.036324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:46.052635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256f38d-7c57-4cfe-a59c-423322ddf388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:46.053421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256f38d-7c57-4cfe-a59c-423322ddf388 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:46.053706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256f38d-7c57-4cfe-a59c-423322ddf388 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:46.053984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256f38d-7c57-4cfe-a59c-423322ddf388 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:46.063814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256f38d-7c57-4cfe-a59c-423322ddf388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-4256f38d-7c57-4cfe-a59c-423322ddf388 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:27:46.071699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:46.072702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:46.074072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 641/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jul 14 11:27:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:46.079473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0711740-7672-4cf8-8feb-905c0b7fbd14 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:46.161682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:46.178785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0711740-7672-4cf8-8feb-905c0b7fbd14 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:46.216622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2179323-3550-4b23-91d7-8d5f197bccf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:46.218092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2179323-3550-4b23-91d7-8d5f197bccf9 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:46.218272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2179323-3550-4b23-91d7-8d5f197bccf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:46.218552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2179323-3550-4b23-91d7-8d5f197bccf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:46.224000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2179323-3550-4b23-91d7-8d5f197bccf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:46.224000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2179323-3550-4b23-91d7-8d5f197bccf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:46.224000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2179323-3550-4b23-91d7-8d5f197bccf9 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:46.229701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 636/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jul 14 11:27:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:46.235450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256f38d-7c57-4cfe-a59c-423322ddf388 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:46.334567 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256f38d-7c57-4cfe-a59c-423322ddf388 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:46.370147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c162729-9f52-4484-b20f-ef013d7504fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:46.370968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c162729-9f52-4484-b20f-ef013d7504fd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:46.371249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c162729-9f52-4484-b20f-ef013d7504fd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:46.371521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c162729-9f52-4484-b20f-ef013d7504fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:46.382975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c162729-9f52-4484-b20f-ef013d7504fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:46.384428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c162729-9f52-4484-b20f-ef013d7504fd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-6c162729-9f52-4484-b20f-ef013d7504fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:46.404197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:46.405253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:46.406170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:46.406170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:46.406170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:46.406170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:46.406170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 642/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:46.411461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2179323-3550-4b23-91d7-8d5f197bccf9 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:46.510517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2179323-3550-4b23-91d7-8d5f197bccf9 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:46.545477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4e8c0-59bf-4442-b09e-b60539aa603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:46.546144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4e8c0-59bf-4442-b09e-b60539aa603d None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:46.546425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4e8c0-59bf-4442-b09e-b60539aa603d None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:46.546699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4e8c0-59bf-4442-b09e-b60539aa603d None admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:46.547422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4e8c0-59bf-4442-b09e-b60539aa603d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:46.547726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4e8c0-59bf-4442-b09e-b60539aa603d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:46.549100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4e8c0-59bf-4442-b09e-b60539aa603d None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:46.561765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 637/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:46.567856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c162729-9f52-4484-b20f-ef013d7504fd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:46.598861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:46.666969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c162729-9f52-4484-b20f-ef013d7504fd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:46.704904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601bb5a-513e-406e-97c9-0b19badd0ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:46.705725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601bb5a-513e-406e-97c9-0b19badd0ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:46.706004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601bb5a-513e-406e-97c9-0b19badd0ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:46.706275 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601bb5a-513e-406e-97c9-0b19badd0ba0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:46.713272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601bb5a-513e-406e-97c9-0b19badd0ba0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:46.714724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601bb5a-513e-406e-97c9-0b19badd0ba0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-a601bb5a-513e-406e-97c9-0b19badd0ba0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:27:46.729081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:46.734391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:46.735635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 643/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:27:46.736818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4e8c0-59bf-4442-b09e-b60539aa603d None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:46.740572 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:46.840160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4e8c0-59bf-4442-b09e-b60539aa603d None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:46.874196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdc857-f62b-423f-96ef-e1a17a42dc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:46.874780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc857-f62b-423f-96ef-e1a17a42dc16 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:46.875058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc857-f62b-423f-96ef-e1a17a42dc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:46.875327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc857-f62b-423f-96ef-e1a17a42dc16 None admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:46.876166 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdc857-f62b-423f-96ef-e1a17a42dc16 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:46.876468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdc857-f62b-423f-96ef-e1a17a42dc16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:46.877917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdc857-f62b-423f-96ef-e1a17a42dc16 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:46.894751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 638/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jul 14 11:27:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:46.900718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601bb5a-513e-406e-97c9-0b19badd0ba0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:46.990515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:47.000352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601bb5a-513e-406e-97c9-0b19badd0ba0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:47.033029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5f093-fb95-44ca-ba9a-3842f996f501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:47.033699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5f093-fb95-44ca-ba9a-3842f996f501 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:47.033982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5f093-fb95-44ca-ba9a-3842f996f501 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:47.034254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5f093-fb95-44ca-ba9a-3842f996f501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:47.101256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5f093-fb95-44ca-ba9a-3842f996f501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:47.102698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5f093-fb95-44ca-ba9a-3842f996f501 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:47.194272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:47.203949 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 644/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jul 14 11:27:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 11:27:47.209902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdc857-f62b-423f-96ef-e1a17a42dc16 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:47.270130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:47.308780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdc857-f62b-423f-96ef-e1a17a42dc16 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:47.340589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472fab4-8739-4fb8-b6f8-98a0de028d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:47.341401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472fab4-8739-4fb8-b6f8-98a0de028d56 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:47.342210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472fab4-8739-4fb8-b6f8-98a0de028d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:47.342474 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472fab4-8739-4fb8-b6f8-98a0de028d56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:47.352724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472fab4-8739-4fb8-b6f8-98a0de028d56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:47.354217 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472fab4-8739-4fb8-b6f8-98a0de028d56 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-6472fab4-8739-4fb8-b6f8-98a0de028d56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:27:47.371270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:47.372347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:47.373677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:27:47.373677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:27:47.373677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:27:47.373677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:47.373677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 639/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:47.378293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5f093-fb95-44ca-ba9a-3842f996f501 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:47.481327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5f093-fb95-44ca-ba9a-3842f996f501 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:47.515479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48126999-26fe-4caf-8e49-0677bcf5f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:47.516137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48126999-26fe-4caf-8e49-0677bcf5f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:47.516417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48126999-26fe-4caf-8e49-0677bcf5f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:47.516689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48126999-26fe-4caf-8e49-0677bcf5f0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:47.517432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48126999-26fe-4caf-8e49-0677bcf5f0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:47.517735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48126999-26fe-4caf-8e49-0677bcf5f0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:47.519088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48126999-26fe-4caf-8e49-0677bcf5f0a7 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:47.530835 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 645/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jul 14 11:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:47.538489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472fab4-8739-4fb8-b6f8-98a0de028d56 None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:47.641821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472fab4-8739-4fb8-b6f8-98a0de028d56 None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:47.655838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:47.675057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd8aa5-9fd6-48ce-8243-32483bd07d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:47.675846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd8aa5-9fd6-48ce-8243-32483bd07d3a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:47.676133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd8aa5-9fd6-48ce-8243-32483bd07d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:47.676409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd8aa5-9fd6-48ce-8243-32483bd07d3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:47.683219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd8aa5-9fd6-48ce-8243-32483bd07d3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:47.684676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd8aa5-9fd6-48ce-8243-32483bd07d3a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application [None req-4bfd8aa5-9fd6-48ce-8243-32483bd07d3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:27:47.699397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:47.700533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:47.702227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:27:47.702227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:27:47.702227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:27:47.702227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:27:47.702227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:27:47.702227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:27:47.702227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:27:47.702227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: ERROR keystone.server.flask.application Jul 14 11:27:47.702227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 640/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jul 14 11:27:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:47.706964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48126999-26fe-4caf-8e49-0677bcf5f0a7 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:47.756374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:47.805811 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48126999-26fe-4caf-8e49-0677bcf5f0a7 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:47.846336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ce3ab-e0dc-43c3-929b-a7eb426319f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:47.847723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ce3ab-e0dc-43c3-929b-a7eb426319f9 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:47.847997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ce3ab-e0dc-43c3-929b-a7eb426319f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:47.848272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ce3ab-e0dc-43c3-929b-a7eb426319f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:47.849078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ce3ab-e0dc-43c3-929b-a7eb426319f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:47.849416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ce3ab-e0dc-43c3-929b-a7eb426319f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:47.850880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ce3ab-e0dc-43c3-929b-a7eb426319f9 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:47.866792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 646/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jul 14 11:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:47.873717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd8aa5-9fd6-48ce-8243-32483bd07d3a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:47.972216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd8aa5-9fd6-48ce-8243-32483bd07d3a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:48.007413 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f222726-34b1-44b9-bc7f-ac35a249c534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:48.007995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222726-34b1-44b9-bc7f-ac35a249c534 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:48.008277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222726-34b1-44b9-bc7f-ac35a249c534 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:48.008551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222726-34b1-44b9-bc7f-ac35a249c534 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:48.009409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f222726-34b1-44b9-bc7f-ac35a249c534 None admin] RBAC: Adding query filter params (role.id=641b0deca0e94bb6bd15d6e839352262, scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a, user.id=fa879d581c6d44e18a8703e66ffbd549) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:48.009728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f222726-34b1-44b9-bc7f-ac35a249c534 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:48.011149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f222726-34b1-44b9-bc7f-ac35a249c534 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:48.017055 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 641/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jul 14 11:27:47 2022] GET /identity/v3/role_assignments?user.id=fa879d581c6d44e18a8703e66ffbd549&scope.project.id=69cec1c5d7884e1b94e44e7b586ef19a&role.id=641b0deca0e94bb6bd15d6e839352262 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:50.083329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681db49-117c-4a67-8314-a38a450f39dd None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:50.083846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681db49-117c-4a67-8314-a38a450f39dd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:50.083846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681db49-117c-4a67-8314-a38a450f39dd None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:50.085181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 647/1288] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:50.090520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71b05e-c93d-4948-b5af-705473c27641 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:50.090927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71b05e-c93d-4948-b5af-705473c27641 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:50.091075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71b05e-c93d-4948-b5af-705473c27641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:50.108058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-1f71b05e-c93d-4948-b5af-705473c27641 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:50.109386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1f71b05e-c93d-4948-b5af-705473c27641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:50.138645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:50.174252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 642/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:27:50.196577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681db49-117c-4a67-8314-a38a450f39dd None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:50.295555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681db49-117c-4a67-8314-a38a450f39dd None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:50.306041 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:50.329251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d95d6-ebc0-4838-b479-61d1218cf4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:50.330030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d95d6-ebc0-4838-b479-61d1218cf4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:50.330309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d95d6-ebc0-4838-b479-61d1218cf4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:50.330586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d95d6-ebc0-4838-b479-61d1218cf4d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:50.331230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d95d6-ebc0-4838-b479-61d1218cf4d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:50.332592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d95d6-ebc0-4838-b479-61d1218cf4d0 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b49d95d6-ebc0-4838-b479-61d1218cf4d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:50.344059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:50.345419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:50.346554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 648/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jul 14 11:27:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:50.351673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71b05e-c93d-4948-b5af-705473c27641 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:50.454206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71b05e-c93d-4948-b5af-705473c27641 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:50.488569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fe88d-20af-4092-96ff-720df172a498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:50.489271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fe88d-20af-4092-96ff-720df172a498 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:50.489559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fe88d-20af-4092-96ff-720df172a498 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:50.489837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fe88d-20af-4092-96ff-720df172a498 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:50.490548 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fe88d-20af-4092-96ff-720df172a498 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:50.490858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fe88d-20af-4092-96ff-720df172a498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:50.492240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fe88d-20af-4092-96ff-720df172a498 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:50.497094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 643/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jul 14 11:27:50 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:50.502960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d95d6-ebc0-4838-b479-61d1218cf4d0 None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:50.607966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d95d6-ebc0-4838-b479-61d1218cf4d0 None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:50.640561 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68ffcb-5c6e-4e93-8bae-fcd442628134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:50.641253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68ffcb-5c6e-4e93-8bae-fcd442628134 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:50.641528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68ffcb-5c6e-4e93-8bae-fcd442628134 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:50.641807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68ffcb-5c6e-4e93-8bae-fcd442628134 None admin] PATH_INFO: `/v3/services` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:50.642512 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68ffcb-5c6e-4e93-8bae-fcd442628134 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:50.642818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68ffcb-5c6e-4e93-8bae-fcd442628134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:50.644183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68ffcb-5c6e-4e93-8bae-fcd442628134 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:50.648797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 649/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jul 14 11:27:50 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:27:52.707075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db0c3-ad83-4730-a4b1-aac3f0cdcd11 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:52.707625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db0c3-ad83-4730-a4b1-aac3f0cdcd11 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:52.707625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db0c3-ad83-4730-a4b1-aac3f0cdcd11 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:52.709071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 644/1293] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:52.714297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7721d-ee67-43c6-a396-8f55ce61659a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:52.714686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7721d-ee67-43c6-a396-8f55ce61659a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:52.714862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7721d-ee67-43c6-a396-8f55ce61659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:52.731984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-64d7721d-ee67-43c6-a396-8f55ce61659a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:52.733362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-64d7721d-ee67-43c6-a396-8f55ce61659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:52.763954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:52.803488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 650/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:27:52.827363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5db0c3-ad83-4730-a4b1-aac3f0cdcd11 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:52.857560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:52.927537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5db0c3-ad83-4730-a4b1-aac3f0cdcd11 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:52.962303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a65c37-f0e9-44fc-b967-6fd4ee51262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:52.962888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a65c37-f0e9-44fc-b967-6fd4ee51262a None admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:52.963171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a65c37-f0e9-44fc-b967-6fd4ee51262a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:52.963446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a65c37-f0e9-44fc-b967-6fd4ee51262a None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:52.964162 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a65c37-f0e9-44fc-b967-6fd4ee51262a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:52.965569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a65c37-f0e9-44fc-b967-6fd4ee51262a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:53.023932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 645/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jul 14 11:27:52 2022] POST /identity/v3/services => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:27:55.037898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db5454-cf67-4c99-8cf7-6240d69ed812 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:55.038447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db5454-cf67-4c99-8cf7-6240d69ed812 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:55.038447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db5454-cf67-4c99-8cf7-6240d69ed812 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:55.039825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 651/1296] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:55.045058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98544c-95c0-4846-b458-1fc188a5263c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:55.055062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98544c-95c0-4846-b458-1fc188a5263c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:55.055062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98544c-95c0-4846-b458-1fc188a5263c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:55.062532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-0b98544c-95c0-4846-b458-1fc188a5263c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:55.063707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0b98544c-95c0-4846-b458-1fc188a5263c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:55.134342 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 646/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:55 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:27:55.157321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db5454-cf67-4c99-8cf7-6240d69ed812 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:55.256881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db5454-cf67-4c99-8cf7-6240d69ed812 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:55.291897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d2433-ae6b-410f-898a-a90c6f84d7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:55.292706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d2433-ae6b-410f-898a-a90c6f84d7bb None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:55.292992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d2433-ae6b-410f-898a-a90c6f84d7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:55.293302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d2433-ae6b-410f-898a-a90c6f84d7bb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:55.293962 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d2433-ae6b-410f-898a-a90c6f84d7bb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:55.295381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d2433-ae6b-410f-898a-a90c6f84d7bb None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-614d2433-ae6b-410f-898a-a90c6f84d7bb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:55.321015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:55.322943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:55.324623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 652/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jul 14 11:27:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:55.326883 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98544c-95c0-4846-b458-1fc188a5263c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:55.336673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:55.418130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98544c-95c0-4846-b458-1fc188a5263c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:55.453372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8eaecf-5a8f-48ac-974f-f62ba8657b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:55.454029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8eaecf-5a8f-48ac-974f-f62ba8657b0a None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:55.454310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8eaecf-5a8f-48ac-974f-f62ba8657b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:55.454579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8eaecf-5a8f-48ac-974f-f62ba8657b0a None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:55.455281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8eaecf-5a8f-48ac-974f-f62ba8657b0a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:55.455603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8eaecf-5a8f-48ac-974f-f62ba8657b0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:55.457003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8eaecf-5a8f-48ac-974f-f62ba8657b0a None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:55.462236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 647/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jul 14 11:27:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:55.468574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d2433-ae6b-410f-898a-a90c6f84d7bb None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:55.501822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:55.568888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d2433-ae6b-410f-898a-a90c6f84d7bb None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:55.603832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f50bd5-15eb-4219-a83e-71e0d1ad9a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:55.604415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f50bd5-15eb-4219-a83e-71e0d1ad9a74 None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:55.604708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f50bd5-15eb-4219-a83e-71e0d1ad9a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:55.605013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f50bd5-15eb-4219-a83e-71e0d1ad9a74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:55.605840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f50bd5-15eb-4219-a83e-71e0d1ad9a74 None admin] RBAC: Adding query filter params (interface=public, service_id=ed80b1e1cba1416e8eb32ddf8c951a4d, region_id=RegionOne) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:55.606147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f50bd5-15eb-4219-a83e-71e0d1ad9a74 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:55.607539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f50bd5-15eb-4219-a83e-71e0d1ad9a74 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:55.612494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 653/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jul 14 11:27:55 2022] GET /identity/v3/endpoints?service_id=ed80b1e1cba1416e8eb32ddf8c951a4d&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:27:55.618373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8eaecf-5a8f-48ac-974f-f62ba8657b0a None admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:55.705233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:55.723322 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8eaecf-5a8f-48ac-974f-f62ba8657b0a None admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:55.757492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614ccf5-79f0-4eef-a46c-2a99020894b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:55.759495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614ccf5-79f0-4eef-a46c-2a99020894b2 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:55.759495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614ccf5-79f0-4eef-a46c-2a99020894b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:55.759495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614ccf5-79f0-4eef-a46c-2a99020894b2 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:55.759495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614ccf5-79f0-4eef-a46c-2a99020894b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:55.760708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614ccf5-79f0-4eef-a46c-2a99020894b2 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:55.767954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 648/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jul 14 11:27:55 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:27:57.849381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ece42-7613-46cd-a8e7-d4455d77a448 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:57.849904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ece42-7613-46cd-a8e7-d4455d77a448 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:57.850016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ece42-7613-46cd-a8e7-d4455d77a448 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:57.851299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 654/1302] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:27:57.856699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed50bc-2b3e-4a00-8565-371d74e65ab5 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:57.858168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed50bc-2b3e-4a00-8565-371d74e65ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:57.858168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed50bc-2b3e-4a00-8565-371d74e65ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:57.873934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e4ed50bc-2b3e-4a00-8565-371d74e65ab5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:57.875129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e4ed50bc-2b3e-4a00-8565-371d74e65ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:57.940194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 649/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:27:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:27:57.963539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ece42-7613-46cd-a8e7-d4455d77a448 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:58.059773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:58.067654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ece42-7613-46cd-a8e7-d4455d77a448 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:58.103424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc75c4-c6aa-4a7d-874a-6602d2bb18dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:58.104215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc75c4-c6aa-4a7d-874a-6602d2bb18dd None admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:58.104499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc75c4-c6aa-4a7d-874a-6602d2bb18dd None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:58.104774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc75c4-c6aa-4a7d-874a-6602d2bb18dd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:58.105455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc75c4-c6aa-4a7d-874a-6602d2bb18dd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:58.106859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc75c4-c6aa-4a7d-874a-6602d2bb18dd None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application [None req-b6bc75c4-c6aa-4a7d-874a-6602d2bb18dd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:27:58.118479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:27:58.119832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application with Lock( Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: ERROR keystone.server.flask.application Jul 14 11:27:58.120961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 655/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jul 14 11:27:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:27:58.126474 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed50bc-2b3e-4a00-8565-371d74e65ab5 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:58.202014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:58.224531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed50bc-2b3e-4a00-8565-371d74e65ab5 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:58.257396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d42fc4-20a0-4502-93bf-733c17b149a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:58.258416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d42fc4-20a0-4502-93bf-733c17b149a3 None admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:58.259131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d42fc4-20a0-4502-93bf-733c17b149a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:58.259409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d42fc4-20a0-4502-93bf-733c17b149a3 None admin] PATH_INFO: `/v3/services` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:58.260154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d42fc4-20a0-4502-93bf-733c17b149a3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:58.260461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d42fc4-20a0-4502-93bf-733c17b149a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:58.261894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d42fc4-20a0-4502-93bf-733c17b149a3 None admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:58.272791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 650/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jul 14 11:27:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:27:58.273065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc75c4-c6aa-4a7d-874a-6602d2bb18dd None admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:58.371768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc75c4-c6aa-4a7d-874a-6602d2bb18dd None admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:58.407073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71427260-20cd-44e5-bcef-7d96cd55d831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:58.407734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71427260-20cd-44e5-bcef-7d96cd55d831 None admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:58.408016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71427260-20cd-44e5-bcef-7d96cd55d831 None admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:58.408294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71427260-20cd-44e5-bcef-7d96cd55d831 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:58.409042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71427260-20cd-44e5-bcef-7d96cd55d831 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:58.410554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71427260-20cd-44e5-bcef-7d96cd55d831 None admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:58.521936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 656/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jul 14 11:27:58 2022] POST /identity/v3/endpoints => generated 331 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:28:08.264043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=30 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:28:08.516988 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:17.678983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3d37f-f235-44ee-ae7d-0dc9fe774b82 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:17.680177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3d37f-f235-44ee-ae7d-0dc9fe774b82 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:17.680606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3d37f-f235-44ee-ae7d-0dc9fe774b82 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:17.689134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4906d-0d8f-4722-b57b-f128ff19a8d1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:17.690026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4906d-0d8f-4722-b57b-f128ff19a8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:17.690450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4906d-0d8f-4722-b57b-f128ff19a8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:17.694214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 651/1307] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:17 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:17.778726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-69a4906d-0d8f-4722-b57b-f128ff19a8d1 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:17.780227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-69a4906d-0d8f-4722-b57b-f128ff19a8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:17.941668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 657/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:35:17.980633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6d1a6-0dd5-4a96-8b21-90fe673170ac None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:17.980943 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6d1a6-0dd5-4a96-8b21-90fe673170ac None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:17.981403 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6d1a6-0dd5-4a96-8b21-90fe673170ac None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:17.983213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 652/1309] 10.222.0.40 () {54 vars in 964 bytes} [Thu Jul 14 11:35:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:17.999260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f0503-9c06-4b9d-9fb0-bad9dca706ec None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:17.999512 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f0503-9c06-4b9d-9fb0-bad9dca706ec None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:17.999788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f0503-9c06-4b9d-9fb0-bad9dca706ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:18.114521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-111f0503-9c06-4b9d-9fb0-bad9dca706ec None None] MFA Rules not processed for user `481ef56b1cfa465a8efc59736a34e412`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:18.114521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-111f0503-9c06-4b9d-9fb0-bad9dca706ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:18.139813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:18.218518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 658/1310] 10.222.0.40 () {58 vars in 1061 bytes} [Thu Jul 14 11:35:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:18.227518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6d1a6-0dd5-4a96-8b21-90fe673170ac None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:18.230060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6dc6d1a6-0dd5-4a96-8b21-90fe673170ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:18.354645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6d1a6-0dd5-4a96-8b21-90fe673170ac None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:18.436215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95b654-7fd3-4627-829b-3c256b89b156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:18.436980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95b654-7fd3-4627-829b-3c256b89b156 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:18.437353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95b654-7fd3-4627-829b-3c256b89b156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:18.437697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95b654-7fd3-4627-829b-3c256b89b156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:18.439429 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ce95b654-7fd3-4627-829b-3c256b89b156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:18.499238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:18.556996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95b654-7fd3-4627-829b-3c256b89b156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:18.558475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95b654-7fd3-4627-829b-3c256b89b156 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:18.578208 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 653/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:35:20.996128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071a35b-2423-4857-b3c4-d773f749bc5c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:20.996670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071a35b-2423-4857-b3c4-d773f749bc5c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:20.996670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071a35b-2423-4857-b3c4-d773f749bc5c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:20.998145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 659/1312] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:21.003250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be12e7f-30b0-4304-88cd-728eb2b7a9e1 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:21.003793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be12e7f-30b0-4304-88cd-728eb2b7a9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:21.003793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be12e7f-30b0-4304-88cd-728eb2b7a9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:21.082083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-1be12e7f-30b0-4304-88cd-728eb2b7a9e1 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:21.083218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1be12e7f-30b0-4304-88cd-728eb2b7a9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:21.202108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 654/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:21.259090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb2982-433d-4d48-b3e8-bf47d2b82666 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:21.259390 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb2982-433d-4d48-b3e8-bf47d2b82666 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:21.259684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb2982-433d-4d48-b3e8-bf47d2b82666 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:21.261625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 660/1314] 10.222.0.40 () {54 vars in 964 bytes} [Thu Jul 14 11:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:21.268807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4421b-7a93-4564-a413-c1b99d96139d None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:21.269149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4421b-7a93-4564-a413-c1b99d96139d None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:21.269595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4421b-7a93-4564-a413-c1b99d96139d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:21.281867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:21.362907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-27e4421b-7a93-4564-a413-c1b99d96139d None None] MFA Rules not processed for user `481ef56b1cfa465a8efc59736a34e412`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:21.364219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-27e4421b-7a93-4564-a413-c1b99d96139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:21.449692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 655/1315] 10.222.0.40 () {58 vars in 1061 bytes} [Thu Jul 14 11:35:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:21.458449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb2982-433d-4d48-b3e8-bf47d2b82666 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:21.459947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1ebb2982-433d-4d48-b3e8-bf47d2b82666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:21.570522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb2982-433d-4d48-b3e8-bf47d2b82666 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:21.644877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:21.647598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaadfb-3616-4c22-b67b-326a20e0f7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:21.648329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaadfb-3616-4c22-b67b-326a20e0f7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:21.648606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaadfb-3616-4c22-b67b-326a20e0f7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:21.648882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaadfb-3616-4c22-b67b-326a20e0f7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:21.651167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1adaadfb-3616-4c22-b67b-326a20e0f7f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:21.764226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaadfb-3616-4c22-b67b-326a20e0f7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:21.765711 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaadfb-3616-4c22-b67b-326a20e0f7f8 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:21.784041 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 661/1316] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:24.597314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445634a-88be-427f-9fbb-e569d68cf872 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:24.597850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445634a-88be-427f-9fbb-e569d68cf872 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:24.597967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445634a-88be-427f-9fbb-e569d68cf872 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:24.599298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 656/1317] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:24.604414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fa95e-fb5d-4308-802b-ce841019ae8d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:24.604793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fa95e-fb5d-4308-802b-ce841019ae8d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:24.604969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fa95e-fb5d-4308-802b-ce841019ae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:24.667586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-476fa95e-fb5d-4308-802b-ce841019ae8d None None] MFA Rules not processed for user `afd88ca575e04b44b3284ec86097cb2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:24.668798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-476fa95e-fb5d-4308-802b-ce841019ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:24.720086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:24.765244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 662/1318] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:24.805093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445634a-88be-427f-9fbb-e569d68cf872 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:24.807177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f445634a-88be-427f-9fbb-e569d68cf872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:24.915750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445634a-88be-427f-9fbb-e569d68cf872 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:24.918545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:24.988004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151958f2-4e23-4017-acc6-cadb0f6acaea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:24.988754 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151958f2-4e23-4017-acc6-cadb0f6acaea service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:24.989038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151958f2-4e23-4017-acc6-cadb0f6acaea service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:24.989341 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151958f2-4e23-4017-acc6-cadb0f6acaea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:24.990993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-151958f2-4e23-4017-acc6-cadb0f6acaea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:25.096737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151958f2-4e23-4017-acc6-cadb0f6acaea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:25.098223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151958f2-4e23-4017-acc6-cadb0f6acaea service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:25.116614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 657/1319] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:24 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:28.100097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3421ad1-f042-41c9-9b87-1ed899c74c64 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:28.100660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3421ad1-f042-41c9-9b87-1ed899c74c64 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:28.100660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3421ad1-f042-41c9-9b87-1ed899c74c64 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:28.102082 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 663/1320] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:28.107326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b860-9f41-4b95-ad53-a92230edb94e None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:28.107756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b860-9f41-4b95-ad53-a92230edb94e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:28.107869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b860-9f41-4b95-ad53-a92230edb94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:28.148690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:28.173824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9662b860-9f41-4b95-ad53-a92230edb94e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:28.175044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9662b860-9f41-4b95-ad53-a92230edb94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:28.270719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 658/1321] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:28.309504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3421ad1-f042-41c9-9b87-1ed899c74c64 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:28.311036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a3421ad1-f042-41c9-9b87-1ed899c74c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:28.413700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3421ad1-f042-41c9-9b87-1ed899c74c64 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:28.485886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a358f95-43c3-435c-b252-eb920aad6b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:28.486524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a358f95-43c3-435c-b252-eb920aad6b04 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:28.486805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a358f95-43c3-435c-b252-eb920aad6b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:28.487075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a358f95-43c3-435c-b252-eb920aad6b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:28.488708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4a358f95-43c3-435c-b252-eb920aad6b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:28.496757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:28.597075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a358f95-43c3-435c-b252-eb920aad6b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:28.598546 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a358f95-43c3-435c-b252-eb920aad6b04 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:28.616146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 664/1322] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:31.836432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b2bb6-d054-4083-9e16-36001a90ed8b None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:31.836995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b2bb6-d054-4083-9e16-36001a90ed8b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:31.838032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b2bb6-d054-4083-9e16-36001a90ed8b None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:31.839795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 659/1323] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:35:31.845877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6652217-5bb0-4680-b62d-ff087d15bc59 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:31.846471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6652217-5bb0-4680-b62d-ff087d15bc59 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:31.846471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6652217-5bb0-4680-b62d-ff087d15bc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:31.913713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a6652217-5bb0-4680-b62d-ff087d15bc59 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:31.914916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a6652217-5bb0-4680-b62d-ff087d15bc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:31.976088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:32.015259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 665/1324] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:32.052241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b2bb6-d054-4083-9e16-36001a90ed8b None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:32.053825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-000b2bb6-d054-4083-9e16-36001a90ed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:32.169191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b2bb6-d054-4083-9e16-36001a90ed8b None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:32.185799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:32.252132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49b6c0-fd59-467e-9799-c1c958f71d91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:32.252713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49b6c0-fd59-467e-9799-c1c958f71d91 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:32.252996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49b6c0-fd59-467e-9799-c1c958f71d91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:32.253297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49b6c0-fd59-467e-9799-c1c958f71d91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:32.255033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2f49b6c0-fd59-467e-9799-c1c958f71d91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:32.371436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49b6c0-fd59-467e-9799-c1c958f71d91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:32.373006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49b6c0-fd59-467e-9799-c1c958f71d91 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:32.392850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 660/1325] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:34.516713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31615427-f852-40f2-83c0-796f73d06c2a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:34.517318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31615427-f852-40f2-83c0-796f73d06c2a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:34.517318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31615427-f852-40f2-83c0-796f73d06c2a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:34.518686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 666/1326] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:34.523856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84858943-c283-48f4-aff7-508ee09fab33 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:34.524279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84858943-c283-48f4-aff7-508ee09fab33 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:34.524432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84858943-c283-48f4-aff7-508ee09fab33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:34.574081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:34.596378 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-84858943-c283-48f4-aff7-508ee09fab33 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:34.597618 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-84858943-c283-48f4-aff7-508ee09fab33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:34.696225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 661/1327] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:34.731808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31615427-f852-40f2-83c0-796f73d06c2a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:34.733413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-31615427-f852-40f2-83c0-796f73d06c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:34.838601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31615427-f852-40f2-83c0-796f73d06c2a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:34.918626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6fa91-38b7-4eea-8d12-d79fc3d19d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:34.919274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6fa91-38b7-4eea-8d12-d79fc3d19d62 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:34.919558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6fa91-38b7-4eea-8d12-d79fc3d19d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:34.919827 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6fa91-38b7-4eea-8d12-d79fc3d19d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:34.921765 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1db6fa91-38b7-4eea-8d12-d79fc3d19d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:34.943450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:35.041035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6fa91-38b7-4eea-8d12-d79fc3d19d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:35.042532 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6fa91-38b7-4eea-8d12-d79fc3d19d62 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:35.061390 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 667/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:37.208732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815d705-d376-435f-9b87-c4d351978af1 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:37.209334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815d705-d376-435f-9b87-c4d351978af1 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:37.209334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815d705-d376-435f-9b87-c4d351978af1 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:37.210613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 662/1329] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:37.216249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2f7e-db41-4c04-b2c2-b6b407a42e89 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:37.216781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2f7e-db41-4c04-b2c2-b6b407a42e89 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:37.216977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2f7e-db41-4c04-b2c2-b6b407a42e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:37.289088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ea7c2f7e-db41-4c04-b2c2-b6b407a42e89 None None] MFA Rules not processed for user `afd88ca575e04b44b3284ec86097cb2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:37.290943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ea7c2f7e-db41-4c04-b2c2-b6b407a42e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:37.358456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:37.392425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 668/1330] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:37.431107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815d705-d376-435f-9b87-c4d351978af1 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:37.432729 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4815d705-d376-435f-9b87-c4d351978af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:37.534591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815d705-d376-435f-9b87-c4d351978af1 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:37.555155 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:37.610213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff47ad6-402e-498f-bc99-446f7048089b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:37.610870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47ad6-402e-498f-bc99-446f7048089b service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:37.611149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47ad6-402e-498f-bc99-446f7048089b service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:37.611418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47ad6-402e-498f-bc99-446f7048089b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:37.613092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0ff47ad6-402e-498f-bc99-446f7048089b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:37.724959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47ad6-402e-498f-bc99-446f7048089b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:37.726506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47ad6-402e-498f-bc99-446f7048089b service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:37.744988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 663/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:40.256607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94e1f1-0528-41e0-b912-01d7589da655 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:40.257188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94e1f1-0528-41e0-b912-01d7589da655 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:40.257188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94e1f1-0528-41e0-b912-01d7589da655 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:40.259098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 669/1332] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:40.264201 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b9e6-32ca-421c-a52f-4cf0ac2769d5 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:40.264638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b9e6-32ca-421c-a52f-4cf0ac2769d5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:40.264744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b9e6-32ca-421c-a52f-4cf0ac2769d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:40.326713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:40.329613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e346b9e6-32ca-421c-a52f-4cf0ac2769d5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:40.330818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e346b9e6-32ca-421c-a52f-4cf0ac2769d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:40.434717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 664/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:40.470306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94e1f1-0528-41e0-b912-01d7589da655 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:40.471838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5e94e1f1-0528-41e0-b912-01d7589da655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:40.573929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94e1f1-0528-41e0-b912-01d7589da655 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:40.653910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfc657-33dd-4cce-b818-8812f027a508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:40.654550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfc657-33dd-4cce-b818-8812f027a508 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:40.654828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfc657-33dd-4cce-b818-8812f027a508 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:40.655103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfc657-33dd-4cce-b818-8812f027a508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:40.656729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-85cfc657-33dd-4cce-b818-8812f027a508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:40.702350 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:40.812102 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfc657-33dd-4cce-b818-8812f027a508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:40.813728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfc657-33dd-4cce-b818-8812f027a508 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:40.831789 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 670/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:35:42.923316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b08027-480c-4692-b18e-e4aefd8fb6e4 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:42.923899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b08027-480c-4692-b18e-e4aefd8fb6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:42.923899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b08027-480c-4692-b18e-e4aefd8fb6e4 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:42.925249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 665/1335] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:42.931062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e18c8-3831-423d-8473-fa796861e21e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:42.931478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e18c8-3831-423d-8473-fa796861e21e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:42.931601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e18c8-3831-423d-8473-fa796861e21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:42.994936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f83e18c8-3831-423d-8473-fa796861e21e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:42.996146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f83e18c8-3831-423d-8473-fa796861e21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:43.070893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:43.092890 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 671/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:43.133815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b08027-480c-4692-b18e-e4aefd8fb6e4 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:43.135384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-65b08027-480c-4692-b18e-e4aefd8fb6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:43.239435 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b08027-480c-4692-b18e-e4aefd8fb6e4 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:43.267402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:43.313547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516431c4-a11f-4a53-a256-5163d99954d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:43.314190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516431c4-a11f-4a53-a256-5163d99954d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:43.314470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516431c4-a11f-4a53-a256-5163d99954d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:43.314741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516431c4-a11f-4a53-a256-5163d99954d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:43.316400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-516431c4-a11f-4a53-a256-5163d99954d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:43.425306 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516431c4-a11f-4a53-a256-5163d99954d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:43.426680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516431c4-a11f-4a53-a256-5163d99954d9 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:43.448026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 666/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:46.262296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3392a2-6a9c-4ea3-9162-0fd9e39dc59b None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:46.262831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3392a2-6a9c-4ea3-9162-0fd9e39dc59b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:46.262948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3392a2-6a9c-4ea3-9162-0fd9e39dc59b None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:46.264234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 672/1338] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:46.269595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144565f-31a1-4320-8b80-070c05f6146c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:46.270040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144565f-31a1-4320-8b80-070c05f6146c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:46.270149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144565f-31a1-4320-8b80-070c05f6146c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:46.333626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e144565f-31a1-4320-8b80-070c05f6146c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:46.334830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e144565f-31a1-4320-8b80-070c05f6146c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:46.347705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:46.436975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 667/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:46.476216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3392a2-6a9c-4ea3-9162-0fd9e39dc59b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:46.477804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bd3392a2-6a9c-4ea3-9162-0fd9e39dc59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:46.589088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3392a2-6a9c-4ea3-9162-0fd9e39dc59b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:46.678802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf839b0-45ca-41d8-b872-07e6bab82079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:46.679447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf839b0-45ca-41d8-b872-07e6bab82079 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:46.679730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf839b0-45ca-41d8-b872-07e6bab82079 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:46.680004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf839b0-45ca-41d8-b872-07e6bab82079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:46.681678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ecf839b0-45ca-41d8-b872-07e6bab82079 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:46.719196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:46.796919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf839b0-45ca-41d8-b872-07e6bab82079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:46.798422 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf839b0-45ca-41d8-b872-07e6bab82079 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:46.817960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 673/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:51.232276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e947ab-2bf2-4cad-a4cc-c1ded14723c3 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:51.232844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e947ab-2bf2-4cad-a4cc-c1ded14723c3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:51.232844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e947ab-2bf2-4cad-a4cc-c1ded14723c3 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:51.234402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 668/1341] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:51.240126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e842b-5888-4a21-9587-bf48547a78df None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:51.240715 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e842b-5888-4a21-9587-bf48547a78df None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:51.240715 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e842b-5888-4a21-9587-bf48547a78df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:51.314940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6c9e842b-5888-4a21-9587-bf48547a78df None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:51.316159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6c9e842b-5888-4a21-9587-bf48547a78df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:51.426892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:51.439708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 674/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:51.479750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e947ab-2bf2-4cad-a4cc-c1ded14723c3 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:51.481602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-65e947ab-2bf2-4cad-a4cc-c1ded14723c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:51.590969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e947ab-2bf2-4cad-a4cc-c1ded14723c3 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:51.627865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:51.665098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111adbb-1dda-450f-ba25-2a59fffd44d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:51.665661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111adbb-1dda-450f-ba25-2a59fffd44d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:51.665936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111adbb-1dda-450f-ba25-2a59fffd44d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:51.666205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111adbb-1dda-450f-ba25-2a59fffd44d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:51.667842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5111adbb-1dda-450f-ba25-2a59fffd44d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:51.777717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111adbb-1dda-450f-ba25-2a59fffd44d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:51.779190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111adbb-1dda-450f-ba25-2a59fffd44d7 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:51.797661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 669/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:54.820391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7027ae-849a-4288-aa26-93c290c20e1a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:54.820976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7027ae-849a-4288-aa26-93c290c20e1a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:54.821074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7027ae-849a-4288-aa26-93c290c20e1a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:54.822609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 675/1344] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:35:54.828545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fa7d8-b668-491a-b4da-a6d693e0798b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:54.829048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fa7d8-b668-491a-b4da-a6d693e0798b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:54.829405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fa7d8-b668-491a-b4da-a6d693e0798b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:54.898105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-3f9fa7d8-b668-491a-b4da-a6d693e0798b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:54.900000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3f9fa7d8-b668-491a-b4da-a6d693e0798b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:54.920340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:55.001213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 670/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:35:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:55.039825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7027ae-849a-4288-aa26-93c290c20e1a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:55.041443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2e7027ae-849a-4288-aa26-93c290c20e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:55.143510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7027ae-849a-4288-aa26-93c290c20e1a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:55.216964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be042777-7aa9-4c8e-9d61-a467235ef7f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:55.217636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be042777-7aa9-4c8e-9d61-a467235ef7f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:55.217914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be042777-7aa9-4c8e-9d61-a467235ef7f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:55.218182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be042777-7aa9-4c8e-9d61-a467235ef7f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:55.219794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-be042777-7aa9-4c8e-9d61-a467235ef7f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:55.261561 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:55.327813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be042777-7aa9-4c8e-9d61-a467235ef7f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:55.329201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be042777-7aa9-4c8e-9d61-a467235ef7f2 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:55.349768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 676/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:36:00.003550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794432e7-5efa-4457-9ab9-590d70ac9195 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:00.004081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794432e7-5efa-4457-9ab9-590d70ac9195 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:00.004179 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794432e7-5efa-4457-9ab9-590d70ac9195 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:00.005532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 671/1347] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:00.011545 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ec844-ccad-4889-82ef-e540dee57e79 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:00.012016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ec844-ccad-4889-82ef-e540dee57e79 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:00.012415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ec844-ccad-4889-82ef-e540dee57e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:00.082198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-868ec844-ccad-4889-82ef-e540dee57e79 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:00.083420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-868ec844-ccad-4889-82ef-e540dee57e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:00.184152 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:00.186718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 677/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:36:00.225713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794432e7-5efa-4457-9ab9-590d70ac9195 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:00.227825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-794432e7-5efa-4457-9ab9-590d70ac9195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:00.343546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794432e7-5efa-4457-9ab9-590d70ac9195 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:00.396925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:00.425370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bdfdb-a580-4323-b1a2-388b27b7e4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:00.426030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bdfdb-a580-4323-b1a2-388b27b7e4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:00.426311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bdfdb-a580-4323-b1a2-388b27b7e4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:00.426578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bdfdb-a580-4323-b1a2-388b27b7e4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:00.428337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-560bdfdb-a580-4323-b1a2-388b27b7e4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:00.539761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bdfdb-a580-4323-b1a2-388b27b7e4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:00.541243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bdfdb-a580-4323-b1a2-388b27b7e4a3 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:00.559919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 672/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:05.550797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb61f-e3a2-43b6-9eda-5a68557f94ef None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:05.551298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb61f-e3a2-43b6-9eda-5a68557f94ef None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:05.551490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb61f-e3a2-43b6-9eda-5a68557f94ef None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:05.553300 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 678/1350] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:05.559451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf554a0-b0ac-4369-bc76-4ea991991e5b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:05.560292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf554a0-b0ac-4369-bc76-4ea991991e5b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:05.560721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf554a0-b0ac-4369-bc76-4ea991991e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:05.645241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-cdf554a0-b0ac-4369-bc76-4ea991991e5b None None] MFA Rules not processed for user `fa879d581c6d44e18a8703e66ffbd549`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:05.646706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cdf554a0-b0ac-4369-bc76-4ea991991e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:05.688596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:05.780598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 673/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:36:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:09.831847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f22048-63b9-4614-aa02-7e78184af49c None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:09.833280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f22048-63b9-4614-aa02-7e78184af49c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:09.834048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f22048-63b9-4614-aa02-7e78184af49c None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:09.837274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 679/1352] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:36:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:09.843719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe09c3-86b5-48ba-9a19-9d6774ffe6ff None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:09.843719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe09c3-86b5-48ba-9a19-9d6774ffe6ff None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:09.843719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe09c3-86b5-48ba-9a19-9d6774ffe6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:09.936952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-a9fe09c3-86b5-48ba-9a19-9d6774ffe6ff None None] MFA Rules not processed for user `fa879d581c6d44e18a8703e66ffbd549`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:09.938936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a9fe09c3-86b5-48ba-9a19-9d6774ffe6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:10.019044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:10.057700 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 674/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:36:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:11.576180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e078c-e610-49c9-9677-e03c8dcc9711 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:11.576714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e078c-e610-49c9-9677-e03c8dcc9711 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:11.576833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e078c-e610-49c9-9677-e03c8dcc9711 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:11.578209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 680/1354] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:11.583409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08050d61-4208-4bcf-a734-8760dcf9cfa4 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:11.583847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08050d61-4208-4bcf-a734-8760dcf9cfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:11.584100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08050d61-4208-4bcf-a734-8760dcf9cfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:11.659313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-08050d61-4208-4bcf-a734-8760dcf9cfa4 None None] MFA Rules not processed for user `fa879d581c6d44e18a8703e66ffbd549`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:11.660526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-08050d61-4208-4bcf-a734-8760dcf9cfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:11.778458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 675/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:36:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:15.170735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c40bf-9799-44ad-8c2a-1795fd1380a0 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:15.171262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c40bf-9799-44ad-8c2a-1795fd1380a0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:15.171262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c40bf-9799-44ad-8c2a-1795fd1380a0 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:15.172730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 681/1356] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:15.178387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466eeff-b9e4-46f4-9c99-ab509a9b0f0f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:15.179273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466eeff-b9e4-46f4-9c99-ab509a9b0f0f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:15.179855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466eeff-b9e4-46f4-9c99-ab509a9b0f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:15.227535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:15.262851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-4466eeff-b9e4-46f4-9c99-ab509a9b0f0f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:15.265025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4466eeff-b9e4-46f4-9c99-ab509a9b0f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:15.378914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 676/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:15.435983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68376f33-d69f-40d6-aa85-d5526246d3c1 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:15.436602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68376f33-d69f-40d6-aa85-d5526246d3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:15.437105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68376f33-d69f-40d6-aa85-d5526246d3c1 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:15.439157 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 682/1358] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jul 14 11:36:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:15.449147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fca9f-f2fb-43e3-a1e2-4e07a279e258 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:15.449442 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fca9f-f2fb-43e3-a1e2-4e07a279e258 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:15.449731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fca9f-f2fb-43e3-a1e2-4e07a279e258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:15.539247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9e7fca9f-f2fb-43e3-a1e2-4e07a279e258 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:15.540834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9e7fca9f-f2fb-43e3-a1e2-4e07a279e258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:15.601339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:15.675748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 677/1359] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jul 14 11:36:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:15.684019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68376f33-d69f-40d6-aa85-d5526246d3c1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:15.685830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-68376f33-d69f-40d6-aa85-d5526246d3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:15.853879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68376f33-d69f-40d6-aa85-d5526246d3c1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:15.931069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:15.972053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37faaa33-ac75-44c6-ab2f-23da157a6b57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:15.974316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faaa33-ac75-44c6-ab2f-23da157a6b57 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:15.974834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faaa33-ac75-44c6-ab2f-23da157a6b57 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:15.975323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faaa33-ac75-44c6-ab2f-23da157a6b57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:15.977339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-37faaa33-ac75-44c6-ab2f-23da157a6b57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:16.079422 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37faaa33-ac75-44c6-ab2f-23da157a6b57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:16.081027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37faaa33-ac75-44c6-ab2f-23da157a6b57 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:16.100187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 683/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:18.395811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cb40f-359d-4222-a9b0-d0c0693c8761 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:18.396362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cb40f-359d-4222-a9b0-d0c0693c8761 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:18.396362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cb40f-359d-4222-a9b0-d0c0693c8761 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:18.397706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 678/1361] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:18.403936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10232348-accf-41f1-b52d-9c30329fd8b0 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:18.404408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10232348-accf-41f1-b52d-9c30329fd8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:18.404513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10232348-accf-41f1-b52d-9c30329fd8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:18.480612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-10232348-accf-41f1-b52d-9c30329fd8b0 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:18.481864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-10232348-accf-41f1-b52d-9c30329fd8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:18.522259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:18.588960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 684/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:18.633291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cb40f-359d-4222-a9b0-d0c0693c8761 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:18.634967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-742cb40f-359d-4222-a9b0-d0c0693c8761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:18.824029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cb40f-359d-4222-a9b0-d0c0693c8761 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:18.834541 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:18.977722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77972cde-1b65-4693-ba7a-b54488a0f596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:18.978254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972cde-1b65-4693-ba7a-b54488a0f596 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:18.978538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972cde-1b65-4693-ba7a-b54488a0f596 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:18.978817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972cde-1b65-4693-ba7a-b54488a0f596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:18.980513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-77972cde-1b65-4693-ba7a-b54488a0f596 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:19.077644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972cde-1b65-4693-ba7a-b54488a0f596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:19.079303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972cde-1b65-4693-ba7a-b54488a0f596 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:19.099889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 679/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:21.626179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8773081-98ed-4574-a356-23649fb3ba8f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:21.626723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8773081-98ed-4574-a356-23649fb3ba8f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:21.626723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8773081-98ed-4574-a356-23649fb3ba8f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:21.628617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 685/1364] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:21.634137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e6a35-639f-4027-ad21-f16ed0222867 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:21.634575 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e6a35-639f-4027-ad21-f16ed0222867 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:21.634883 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e6a35-639f-4027-ad21-f16ed0222867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:21.651035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:21.703678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-843e6a35-639f-4027-ad21-f16ed0222867 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:21.704995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-843e6a35-639f-4027-ad21-f16ed0222867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:21.819839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 680/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:36:21.867572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29748192-bc26-43ab-a659-c7adaab071b1 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:21.867858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29748192-bc26-43ab-a659-c7adaab071b1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:21.868159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29748192-bc26-43ab-a659-c7adaab071b1 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:21.871721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 686/1366] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jul 14 11:36:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:21.879433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b93fd1-d402-443c-b0ce-9580904790c4 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:21.879433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b93fd1-d402-443c-b0ce-9580904790c4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:21.879433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b93fd1-d402-443c-b0ce-9580904790c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:21.964819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-75b93fd1-d402-443c-b0ce-9580904790c4 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:21.966100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-75b93fd1-d402-443c-b0ce-9580904790c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:22.017778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:22.109330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 681/1367] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jul 14 11:36:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:22.118837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29748192-bc26-43ab-a659-c7adaab071b1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:22.121094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-29748192-bc26-43ab-a659-c7adaab071b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:22.304138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29748192-bc26-43ab-a659-c7adaab071b1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:22.325875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:22.422938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378f534-1f4d-412c-b3d3-d97e316c8ccd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:22.423582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378f534-1f4d-412c-b3d3-d97e316c8ccd service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:22.423858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378f534-1f4d-412c-b3d3-d97e316c8ccd service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:22.424124 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378f534-1f4d-412c-b3d3-d97e316c8ccd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:22.425768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c378f534-1f4d-412c-b3d3-d97e316c8ccd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:22.527632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378f534-1f4d-412c-b3d3-d97e316c8ccd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:22.530840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378f534-1f4d-412c-b3d3-d97e316c8ccd service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:22.552830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 687/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:23.454261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158071a-0d04-4e2a-b20e-fd0bf6484fb0 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:23.454762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158071a-0d04-4e2a-b20e-fd0bf6484fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:23.454762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158071a-0d04-4e2a-b20e-fd0bf6484fb0 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:23.456161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 682/1369] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:23.462186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ab041-f9c7-4f43-8124-5c4032635c19 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:23.462683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ab041-f9c7-4f43-8124-5c4032635c19 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:23.463351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ab041-f9c7-4f43-8124-5c4032635c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:23.499684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:23.539849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4b4ab041-f9c7-4f43-8124-5c4032635c19 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:23.542076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4b4ab041-f9c7-4f43-8124-5c4032635c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:23.645208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 688/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:23.991916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99997ee-fe49-4736-af07-30c53c282a87 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:23.991916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99997ee-fe49-4736-af07-30c53c282a87 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:23.991916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99997ee-fe49-4736-af07-30c53c282a87 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:23.993364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 683/1371] 10.222.0.40 () {54 vars in 963 bytes} [Thu Jul 14 11:36:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:23.998372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2905c7-2d8b-4774-a523-df61bf72202d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:23.998657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2905c7-2d8b-4774-a523-df61bf72202d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:23.998929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2905c7-2d8b-4774-a523-df61bf72202d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:24.085454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8a2905c7-2d8b-4774-a523-df61bf72202d None None] MFA Rules not processed for user `f12e348368ac42c696623b4a8f781977`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:24.087060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8a2905c7-2d8b-4774-a523-df61bf72202d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:24.141356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:24.172613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 689/1372] 10.222.0.40 () {58 vars in 1060 bytes} [Thu Jul 14 11:36:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:24.180773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99997ee-fe49-4736-af07-30c53c282a87 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:24.182332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b99997ee-fe49-4736-af07-30c53c282a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:24.298650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99997ee-fe49-4736-af07-30c53c282a87 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:24.342841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:24.388108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196fe68-129a-4b8b-81ba-1b220f12b99c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:24.389084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196fe68-129a-4b8b-81ba-1b220f12b99c service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:24.389400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196fe68-129a-4b8b-81ba-1b220f12b99c service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:24.389808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196fe68-129a-4b8b-81ba-1b220f12b99c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:24.391471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a196fe68-129a-4b8b-81ba-1b220f12b99c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:24.521537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196fe68-129a-4b8b-81ba-1b220f12b99c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:24.523129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196fe68-129a-4b8b-81ba-1b220f12b99c service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:24.544854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 684/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:25.075187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168c9ab-b7f6-4987-93ee-85289a3b0969 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:25.075657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168c9ab-b7f6-4987-93ee-85289a3b0969 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:25.075756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168c9ab-b7f6-4987-93ee-85289a3b0969 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:25.077127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 690/1374] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:25.082556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5559d-2b2f-4d07-a6f6-b5c2a8a29fc9 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:25.082848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5559d-2b2f-4d07-a6f6-b5c2a8a29fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:25.083127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5559d-2b2f-4d07-a6f6-b5c2a8a29fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:25.167237 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-b4c5559d-2b2f-4d07-a6f6-b5c2a8a29fc9 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:25.168460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b4c5559d-2b2f-4d07-a6f6-b5c2a8a29fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:25.190839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:25.269524 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 685/1375] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:25.313806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168c9ab-b7f6-4987-93ee-85289a3b0969 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:25.315481 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a168c9ab-b7f6-4987-93ee-85289a3b0969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:25.470211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168c9ab-b7f6-4987-93ee-85289a3b0969 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:25.522279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:25.586404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72473ad9-6e9a-4d23-a9dc-489a4bfed81e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:25.587049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72473ad9-6e9a-4d23-a9dc-489a4bfed81e service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:25.587333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72473ad9-6e9a-4d23-a9dc-489a4bfed81e service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:25.587846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72473ad9-6e9a-4d23-a9dc-489a4bfed81e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:25.589584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-72473ad9-6e9a-4d23-a9dc-489a4bfed81e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:25.681491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72473ad9-6e9a-4d23-a9dc-489a4bfed81e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:25.683115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72473ad9-6e9a-4d23-a9dc-489a4bfed81e service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:25.701696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 691/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:28.057104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015317c-57db-4936-86a3-453e67a959e8 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:28.057665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015317c-57db-4936-86a3-453e67a959e8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:28.057665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015317c-57db-4936-86a3-453e67a959e8 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:28.059164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 686/1377] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:28.064424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57c21-ab13-4168-9537-95cae690a569 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:28.064888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57c21-ab13-4168-9537-95cae690a569 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:28.064986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57c21-ab13-4168-9537-95cae690a569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:28.148698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e5c57c21-ab13-4168-9537-95cae690a569 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:28.150047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e5c57c21-ab13-4168-9537-95cae690a569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:28.156684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:28.294262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 692/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:28.341894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015317c-57db-4936-86a3-453e67a959e8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:28.343411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5015317c-57db-4936-86a3-453e67a959e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:28.505819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015317c-57db-4936-86a3-453e67a959e8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:28.510043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:28.627848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa351042-9ab4-48d3-8950-72183d225aaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:28.628412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa351042-9ab4-48d3-8950-72183d225aaf service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:28.628693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa351042-9ab4-48d3-8950-72183d225aaf service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:28.628964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa351042-9ab4-48d3-8950-72183d225aaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:28.630821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-aa351042-9ab4-48d3-8950-72183d225aaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:28.725454 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa351042-9ab4-48d3-8950-72183d225aaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:28.727702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa351042-9ab4-48d3-8950-72183d225aaf service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:28.747410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 687/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:30.953417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c05ef-f4d5-4b70-bc7a-a13b757f1463 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:30.953977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c05ef-f4d5-4b70-bc7a-a13b757f1463 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:30.953977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c05ef-f4d5-4b70-bc7a-a13b757f1463 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:30.955500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 693/1380] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:30.961000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3ffe-6ee3-4158-9d83-27c972827801 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:30.961476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3ffe-6ee3-4158-9d83-27c972827801 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:30.961567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3ffe-6ee3-4158-9d83-27c972827801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:30.971795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:31.037052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-54de3ffe-6ee3-4158-9d83-27c972827801 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:31.038931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-54de3ffe-6ee3-4158-9d83-27c972827801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:31.151011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 688/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:31.200132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c05ef-f4d5-4b70-bc7a-a13b757f1463 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:31.202192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-139c05ef-f4d5-4b70-bc7a-a13b757f1463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:31.382185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:31.393018 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c05ef-f4d5-4b70-bc7a-a13b757f1463 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:31.515771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fb6e3-e6c0-4038-8439-7ec56b1e7c28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:31.516423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fb6e3-e6c0-4038-8439-7ec56b1e7c28 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:31.516712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fb6e3-e6c0-4038-8439-7ec56b1e7c28 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:31.516993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fb6e3-e6c0-4038-8439-7ec56b1e7c28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:31.518735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e77fb6e3-e6c0-4038-8439-7ec56b1e7c28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:31.625234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fb6e3-e6c0-4038-8439-7ec56b1e7c28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:31.627060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fb6e3-e6c0-4038-8439-7ec56b1e7c28 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:31.639142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:31.650030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 694/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:34.376161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3a259-5620-48e8-9cff-a3026ef2b2c8 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:34.376764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3a259-5620-48e8-9cff-a3026ef2b2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:34.376975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3a259-5620-48e8-9cff-a3026ef2b2c8 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:34.378955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 689/1383] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:34.385877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8602edc-c23f-4daa-832c-557da9fa91e9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:34.386365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8602edc-c23f-4daa-832c-557da9fa91e9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:34.386682 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8602edc-c23f-4daa-832c-557da9fa91e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:34.475978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b8602edc-c23f-4daa-832c-557da9fa91e9 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:34.478249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b8602edc-c23f-4daa-832c-557da9fa91e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:34.582305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 695/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:34.634499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3a259-5620-48e8-9cff-a3026ef2b2c8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:34.635928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f1a3a259-5620-48e8-9cff-a3026ef2b2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:34.723543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:34.799520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3a259-5620-48e8-9cff-a3026ef2b2c8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:34.932638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a7f16-5fd4-4329-a81c-6d6da943bfad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:34.933250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7f16-5fd4-4329-a81c-6d6da943bfad service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:34.933532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7f16-5fd4-4329-a81c-6d6da943bfad service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:34.933808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7f16-5fd4-4329-a81c-6d6da943bfad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:34.935477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-501a7f16-5fd4-4329-a81c-6d6da943bfad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:34.995628 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:35.046010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a7f16-5fd4-4329-a81c-6d6da943bfad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:35.048158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a7f16-5fd4-4329-a81c-6d6da943bfad service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:35.072000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 690/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:37.127964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3f578-b473-4f38-9485-2a803a66e10e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:37.128481 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3f578-b473-4f38-9485-2a803a66e10e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:37.128481 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3f578-b473-4f38-9485-2a803a66e10e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:37.129903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 696/1386] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:37.136103 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a6d1-c15d-4ec3-97e0-ad80ca352eb2 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:37.136560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a6d1-c15d-4ec3-97e0-ad80ca352eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:37.136837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a6d1-c15d-4ec3-97e0-ad80ca352eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:37.207364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-7b57a6d1-c15d-4ec3-97e0-ad80ca352eb2 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:37.208591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7b57a6d1-c15d-4ec3-97e0-ad80ca352eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:37.312247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 691/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:37.367304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3f578-b473-4f38-9485-2a803a66e10e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:37.368813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ced3f578-b473-4f38-9485-2a803a66e10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:37.454877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:37.543540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3f578-b473-4f38-9485-2a803a66e10e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:37.675086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1405c65-09ea-4ee8-b434-0c1bb39fbc5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:37.675888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1405c65-09ea-4ee8-b434-0c1bb39fbc5d service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:37.676249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1405c65-09ea-4ee8-b434-0c1bb39fbc5d service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:37.676606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1405c65-09ea-4ee8-b434-0c1bb39fbc5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:37.681823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d1405c65-09ea-4ee8-b434-0c1bb39fbc5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:37.730925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:37.785740 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1405c65-09ea-4ee8-b434-0c1bb39fbc5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:37.787359 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1405c65-09ea-4ee8-b434-0c1bb39fbc5d service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:37.810895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 697/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:40.057880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07431f-75de-4942-b9d7-6d324e8d1d51 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:40.058483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07431f-75de-4942-b9d7-6d324e8d1d51 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:40.058483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07431f-75de-4942-b9d7-6d324e8d1d51 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:40.059801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 692/1389] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:40.065176 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034dc3-d506-4a40-bfd7-3304fa975cd6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:40.065602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034dc3-d506-4a40-bfd7-3304fa975cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:40.066094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034dc3-d506-4a40-bfd7-3304fa975cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:40.132433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-84034dc3-d506-4a40-bfd7-3304fa975cd6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:40.133686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-84034dc3-d506-4a40-bfd7-3304fa975cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:40.229075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 698/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:40.273702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07431f-75de-4942-b9d7-6d324e8d1d51 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:40.275214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-dd07431f-75de-4942-b9d7-6d324e8d1d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:40.306154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:40.424118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07431f-75de-4942-b9d7-6d324e8d1d51 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:40.521502 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d0d48-318f-49eb-a1aa-87596941a800 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:40.521703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d0d48-318f-49eb-a1aa-87596941a800 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:40.522000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d0d48-318f-49eb-a1aa-87596941a800 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:40.524416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 699/1391] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:36:40.529201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9c8c2-f49a-4bdb-b5f0-3945ff86c331 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:40.529497 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9c8c2-f49a-4bdb-b5f0-3945ff86c331 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:40.529766 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9c8c2-f49a-4bdb-b5f0-3945ff86c331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:40.536957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:40.544159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543e2c3-f9b7-4872-b072-4babd5b25395 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:40.544809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543e2c3-f9b7-4872-b072-4babd5b25395 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:40.545089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543e2c3-f9b7-4872-b072-4babd5b25395 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:40.545390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543e2c3-f9b7-4872-b072-4babd5b25395 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:40.547039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b543e2c3-f9b7-4872-b072-4babd5b25395 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:40.548213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:40.604445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3bd9c8c2-f49a-4bdb-b5f0-3945ff86c331 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:40.605693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3bd9c8c2-f49a-4bdb-b5f0-3945ff86c331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:40.643744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543e2c3-f9b7-4872-b072-4babd5b25395 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:40.645413 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543e2c3-f9b7-4872-b072-4babd5b25395 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:40.668396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 693/1392] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:40.718494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 700/1393] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:40.776558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ab941-e20f-48f8-b1ed-2a37ade9c9ff None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:40.776558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ab941-e20f-48f8-b1ed-2a37ade9c9ff None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:40.776558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ab941-e20f-48f8-b1ed-2a37ade9c9ff None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:40.782810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f122dfa-b897-464a-8312-27868e388e77 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:40.783024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 694/1394] 10.222.0.40 () {54 vars in 962 bytes} [Thu Jul 14 11:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:40.783166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f122dfa-b897-464a-8312-27868e388e77 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:40.783459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f122dfa-b897-464a-8312-27868e388e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:40.863760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2f122dfa-b897-464a-8312-27868e388e77 None None] MFA Rules not processed for user `a1a7213cbf6448b7ba6570a52a4c7c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:40.864959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2f122dfa-b897-464a-8312-27868e388e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:40.901344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:40.949021 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 701/1395] 10.222.0.40 () {58 vars in 1059 bytes} [Thu Jul 14 11:36:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:36:40.956796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ab941-e20f-48f8-b1ed-2a37ade9c9ff None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:40.958488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0b3ab941-e20f-48f8-b1ed-2a37ade9c9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:41.070021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ab941-e20f-48f8-b1ed-2a37ade9c9ff None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:41.083243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:41.144525 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aad73-8aa7-42c4-be4a-218f5ceb990b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1a7213cbf6448b7ba6570a52a4c7c0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:41.145216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aad73-8aa7-42c4-be4a-218f5ceb990b service glance] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:41.145500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aad73-8aa7-42c4-be4a-218f5ceb990b service glance] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:41.145779 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aad73-8aa7-42c4-be4a-218f5ceb990b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:41.148028 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-446aad73-8aa7-42c4-be4a-218f5ceb990b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:41.259275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446aad73-8aa7-42c4-be4a-218f5ceb990b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:41.260751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446aad73-8aa7-42c4-be4a-218f5ceb990b service glance] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:41.279542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 695/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:42.188129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc78ab1-cf74-47df-9d27-892ba1690f9c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:42.188592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc78ab1-cf74-47df-9d27-892ba1690f9c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:42.188722 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc78ab1-cf74-47df-9d27-892ba1690f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:42.252808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6bc78ab1-cf74-47df-9d27-892ba1690f9c None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:42.254024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6bc78ab1-cf74-47df-9d27-892ba1690f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:42.343150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 702/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:36:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:42.389815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f469a-016a-402a-8332-1444ff066186 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:42.390296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f469a-016a-402a-8332-1444ff066186 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:42.390388 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f469a-016a-402a-8332-1444ff066186 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:42.391725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 696/1398] 10.222.0.40 () {54 vars in 942 bytes} [Thu Jul 14 11:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:42.398550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519365b0-31c4-411a-b71b-a8f5b206dde9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:42.398840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519365b0-31c4-411a-b71b-a8f5b206dde9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:42.399118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519365b0-31c4-411a-b71b-a8f5b206dde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:42.458277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:42.478294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-519365b0-31c4-411a-b71b-a8f5b206dde9 None None] MFA Rules not processed for user `42a5ec793f014cefb9d0a0d7258bbb59`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:42.488595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-519365b0-31c4-411a-b71b-a8f5b206dde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:42.586480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 703/1399] 10.222.0.40 () {58 vars in 1039 bytes} [Thu Jul 14 11:36:42 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:42.595870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f469a-016a-402a-8332-1444ff066186 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:42.597351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b12f469a-016a-402a-8332-1444ff066186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:42.643569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:42.741312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f469a-016a-402a-8332-1444ff066186 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:42.931850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28753d-f739-4fd6-a909-c8a8895e5d38 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:42.932068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28753d-f739-4fd6-a909-c8a8895e5d38 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:42.932313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28753d-f739-4fd6-a909-c8a8895e5d38 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:42.933728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 704/1400] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:36:42.938517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52266dae-0540-4e57-87e2-283f3f35ea18 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:42.938808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52266dae-0540-4e57-87e2-283f3f35ea18 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:42.939089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52266dae-0540-4e57-87e2-283f3f35ea18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:42.964042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747770a-d90f-4ae9-acda-40493527909b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:42.964692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747770a-d90f-4ae9-acda-40493527909b service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:42.964966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747770a-d90f-4ae9-acda-40493527909b service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:42.965268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747770a-d90f-4ae9-acda-40493527909b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:42.967025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9747770a-d90f-4ae9-acda-40493527909b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:42.998084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:43.006575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-52266dae-0540-4e57-87e2-283f3f35ea18 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:43.009868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-52266dae-0540-4e57-87e2-283f3f35ea18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:43.066352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747770a-d90f-4ae9-acda-40493527909b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:43.067960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747770a-d90f-4ae9-acda-40493527909b service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:43.071162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:43.076611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 697/1401] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:36:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:36:43.118690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 705/1402] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:36:43.164802 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747770a-d90f-4ae9-acda-40493527909b service swift] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:43.166193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9747770a-d90f-4ae9-acda-40493527909b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:43.256579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747770a-d90f-4ae9-acda-40493527909b service swift] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:43.353426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:43.410879 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec1414-17d8-4d3a-9d83-8fbd9de55cae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:43.411629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1414-17d8-4d3a-9d83-8fbd9de55cae service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:43.411880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1414-17d8-4d3a-9d83-8fbd9de55cae service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:43.412156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1414-17d8-4d3a-9d83-8fbd9de55cae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:43.414587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-94ec1414-17d8-4d3a-9d83-8fbd9de55cae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:43.509111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec1414-17d8-4d3a-9d83-8fbd9de55cae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:43.510759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec1414-17d8-4d3a-9d83-8fbd9de55cae service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:43.530857 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 698/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:46.306629 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ae707-22d9-4a55-9bdb-de385eb6a44a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:46.309188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ae707-22d9-4a55-9bdb-de385eb6a44a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:46.309505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ae707-22d9-4a55-9bdb-de385eb6a44a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:46.311187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 706/1404] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:46.318640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d32a2-7965-497d-b584-a20bdfcd3fb1 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:46.319207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d32a2-7965-497d-b584-a20bdfcd3fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:46.319207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d32a2-7965-497d-b584-a20bdfcd3fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:46.407930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-4b5d32a2-7965-497d-b584-a20bdfcd3fb1 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:46.409335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4b5d32a2-7965-497d-b584-a20bdfcd3fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:46.451459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:46.532902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 699/1405] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:46.581940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ae707-22d9-4a55-9bdb-de385eb6a44a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:46.583532 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6a1ae707-22d9-4a55-9bdb-de385eb6a44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:46.759632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ae707-22d9-4a55-9bdb-de385eb6a44a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:46.811563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:46.894465 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea41f0-2517-4da4-b3bd-02017c007316 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:46.895047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea41f0-2517-4da4-b3bd-02017c007316 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:46.895333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea41f0-2517-4da4-b3bd-02017c007316 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:46.895607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea41f0-2517-4da4-b3bd-02017c007316 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:46.897871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c3ea41f0-2517-4da4-b3bd-02017c007316 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:47.003475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea41f0-2517-4da4-b3bd-02017c007316 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:47.010716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea41f0-2517-4da4-b3bd-02017c007316 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:47.024753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 707/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:49.378513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8985824-a1a0-403e-b2b4-6b55419b76f5 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:49.379089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8985824-a1a0-403e-b2b4-6b55419b76f5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:49.379400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8985824-a1a0-403e-b2b4-6b55419b76f5 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:49.381538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 700/1407] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:36:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:36:49.386954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c0cdd-cd59-4bb9-9eab-55107333224e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:49.387472 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c0cdd-cd59-4bb9-9eab-55107333224e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:49.387565 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c0cdd-cd59-4bb9-9eab-55107333224e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:49.471113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:49.481464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d78c0cdd-cd59-4bb9-9eab-55107333224e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:36:49.482792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d78c0cdd-cd59-4bb9-9eab-55107333224e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:49.609441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 708/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:36:49.668744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8985824-a1a0-403e-b2b4-6b55419b76f5 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:36:49.670609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e8985824-a1a0-403e-b2b4-6b55419b76f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:49.902357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8985824-a1a0-403e-b2b4-6b55419b76f5 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:36:49.924967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:50.043885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45160a79-814e-4696-b5e4-e19e6b2fa1d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:36:50.044561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45160a79-814e-4696-b5e4-e19e6b2fa1d5 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:36:50.044922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45160a79-814e-4696-b5e4-e19e6b2fa1d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:36:50.045067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45160a79-814e-4696-b5e4-e19e6b2fa1d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:36:50.046806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-45160a79-814e-4696-b5e4-e19e6b2fa1d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:36:50.147479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45160a79-814e-4696-b5e4-e19e6b2fa1d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:36:50.149477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45160a79-814e-4696-b5e4-e19e6b2fa1d5 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:36:50.171019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 701/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:00.277664 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=44 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:00.350529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=90 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:06.826212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f5ce0-7918-4a9f-a4d4-db48e490a243 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:06.826727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f5ce0-7918-4a9f-a4d4-db48e490a243 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:06.826848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f5ce0-7918-4a9f-a4d4-db48e490a243 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:06.828136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 709/1410] 10.222.0.40 () {54 vars in 918 bytes} [Thu Jul 14 11:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:06.833176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a913-df95-4a72-9510-dc504d2de522 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:06.833611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a913-df95-4a72-9510-dc504d2de522 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:06.833748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a913-df95-4a72-9510-dc504d2de522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:06.922552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-46b4a913-df95-4a72-9510-dc504d2de522 None None] MFA Rules not processed for user `fa879d581c6d44e18a8703e66ffbd549`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:06.923772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-46b4a913-df95-4a72-9510-dc504d2de522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:07.029399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 702/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Thu Jul 14 11:37:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:07.045187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f8355-e964-4ca0-9cbb-11bf428f0f30 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:07.045495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f8355-e964-4ca0-9cbb-11bf428f0f30 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:07.045778 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f8355-e964-4ca0-9cbb-11bf428f0f30 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:07.047139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 710/1412] 10.222.0.40 () {54 vars in 918 bytes} [Thu Jul 14 11:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:07.051935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53391de0-c265-485f-bfb8-b911a6ba20f9 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:07.052168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53391de0-c265-485f-bfb8-b911a6ba20f9 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:07.052449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53391de0-c265-485f-bfb8-b911a6ba20f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:07.122824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:07.125405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-53391de0-c265-485f-bfb8-b911a6ba20f9 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:07.126632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-53391de0-c265-485f-bfb8-b911a6ba20f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:07.242184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 703/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Thu Jul 14 11:37:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:09.760413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a80101-403e-4927-90ec-e6289229e486 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:09.760989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a80101-403e-4927-90ec-e6289229e486 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:09.760989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a80101-403e-4927-90ec-e6289229e486 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:09.762377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:09.767615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cbc3c-59f9-434c-a924-3b88517b4a42 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:09.768015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cbc3c-59f9-434c-a924-3b88517b4a42 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:09.768176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cbc3c-59f9-434c-a924-3b88517b4a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:09.832503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ef7cbc3c-59f9-434c-a924-3b88517b4a42 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:09.833744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ef7cbc3c-59f9-434c-a924-3b88517b4a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:09.904725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:09.949949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:09.994214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a80101-403e-4927-90ec-e6289229e486 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:09.995779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-71a80101-403e-4927-90ec-e6289229e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:10.141444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a80101-403e-4927-90ec-e6289229e486 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:10.217055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:10.254405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cacec-465a-4f52-8c67-f8ce19280bdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:10.254961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cacec-465a-4f52-8c67-f8ce19280bdf service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:10.255250 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cacec-465a-4f52-8c67-f8ce19280bdf service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:10.255526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cacec-465a-4f52-8c67-f8ce19280bdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:10.257738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0f8cacec-465a-4f52-8c67-f8ce19280bdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:10.345760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cacec-465a-4f52-8c67-f8ce19280bdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:10.347370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cacec-465a-4f52-8c67-f8ce19280bdf service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:10.365791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 712/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:11.265547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e8c5e-2afc-4907-ab31-5dd71b3b06e2 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:11.266097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e8c5e-2afc-4907-ab31-5dd71b3b06e2 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:11.266097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e8c5e-2afc-4907-ab31-5dd71b3b06e2 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:11.267437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:11.272793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eebdf7-8d6d-4362-b09c-7f8a342ddef6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:11.273261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eebdf7-8d6d-4362-b09c-7f8a342ddef6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:11.273397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eebdf7-8d6d-4362-b09c-7f8a342ddef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:11.346361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-95eebdf7-8d6d-4362-b09c-7f8a342ddef6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:11.347567 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-95eebdf7-8d6d-4362-b09c-7f8a342ddef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:11.374993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:11.446015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:11.600601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e8c5e-2afc-4907-ab31-5dd71b3b06e2 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:11.602127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c55e8c5e-2afc-4907-ab31-5dd71b3b06e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:11.746349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e8c5e-2afc-4907-ab31-5dd71b3b06e2 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:11.786185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:11.859459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c614-2c1b-404b-b07e-4dc8989e6501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:11.860023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c614-2c1b-404b-b07e-4dc8989e6501 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:11.860308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c614-2c1b-404b-b07e-4dc8989e6501 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:11.860570 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c614-2c1b-404b-b07e-4dc8989e6501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:11.862243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4640c614-2c1b-404b-b07e-4dc8989e6501 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:11.949425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c614-2c1b-404b-b07e-4dc8989e6501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:11.951027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c614-2c1b-404b-b07e-4dc8989e6501 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:11.969280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 706/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:12.844006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95ae26-aed5-459c-aeab-5aad4c6ad4a7 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:12.844563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95ae26-aed5-459c-aeab-5aad4c6ad4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:12.844563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95ae26-aed5-459c-aeab-5aad4c6ad4a7 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:12.845948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:12.851133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd1feb-9d10-467a-b408-697a0cd518da None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:12.851588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd1feb-9d10-467a-b408-697a0cd518da None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:12.851707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd1feb-9d10-467a-b408-697a0cd518da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:12.905033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:12.926692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-1fcd1feb-9d10-467a-b408-697a0cd518da None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:12.928397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1fcd1feb-9d10-467a-b408-697a0cd518da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:13.025976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:13.183442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95ae26-aed5-459c-aeab-5aad4c6ad4a7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:13.184888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cd95ae26-aed5-459c-aeab-5aad4c6ad4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:13.337896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95ae26-aed5-459c-aeab-5aad4c6ad4a7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:13.351886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:13.470686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ec6c3-93c8-4699-929d-7e82531be196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:13.471268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ec6c3-93c8-4699-929d-7e82531be196 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:13.471547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ec6c3-93c8-4699-929d-7e82531be196 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:13.471817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ec6c3-93c8-4699-929d-7e82531be196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:13.473692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fa7ec6c3-93c8-4699-929d-7e82531be196 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:13.577886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ec6c3-93c8-4699-929d-7e82531be196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:13.579527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ec6c3-93c8-4699-929d-7e82531be196 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:13.597464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 715/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:14.471323 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3041f17-cb7e-404b-8e29-53e3253b7978 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:14.471928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3041f17-cb7e-404b-8e29-53e3253b7978 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:14.471928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3041f17-cb7e-404b-8e29-53e3253b7978 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:14.473266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:14.478863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5b28-0739-40c7-a4ec-03095df9fcf7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:14.479282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5b28-0739-40c7-a4ec-03095df9fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:14.479420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5b28-0739-40c7-a4ec-03095df9fcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:14.497747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:14.554772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b3bb5b28-0739-40c7-a4ec-03095df9fcf7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:14.555982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b3bb5b28-0739-40c7-a4ec-03095df9fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:14.650245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:14.805119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3041f17-cb7e-404b-8e29-53e3253b7978 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:14.809684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b3041f17-cb7e-404b-8e29-53e3253b7978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:14.922379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:14.959123 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3041f17-cb7e-404b-8e29-53e3253b7978 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:15.073254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb96895-77dd-412c-b20b-c9b1a49670b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:15.073838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb96895-77dd-412c-b20b-c9b1a49670b4 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:15.074128 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb96895-77dd-412c-b20b-c9b1a49670b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:15.074405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb96895-77dd-412c-b20b-c9b1a49670b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:15.076058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4bb96895-77dd-412c-b20b-c9b1a49670b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:15.150371 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:15.164043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb96895-77dd-412c-b20b-c9b1a49670b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:15.165674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb96895-77dd-412c-b20b-c9b1a49670b4 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:15.183666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 709/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:16.081319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500067e-0741-46b1-9b04-f92383ce80f0 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:16.082030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500067e-0741-46b1-9b04-f92383ce80f0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:16.082030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500067e-0741-46b1-9b04-f92383ce80f0 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:16.083294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:16.088295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dac3d5-69ab-4674-b10e-b0a9b5d373fd None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:16.088758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dac3d5-69ab-4674-b10e-b0a9b5d373fd None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:16.088861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dac3d5-69ab-4674-b10e-b0a9b5d373fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:16.161615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-06dac3d5-69ab-4674-b10e-b0a9b5d373fd None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:16.163309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-06dac3d5-69ab-4674-b10e-b0a9b5d373fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:16.265269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:16.441844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500067e-0741-46b1-9b04-f92383ce80f0 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:16.444510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4500067e-0741-46b1-9b04-f92383ce80f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:16.527608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:16.588919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500067e-0741-46b1-9b04-f92383ce80f0 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:16.698888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58c6a4-99f9-4e40-abff-20f3aa4c174a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:16.699445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58c6a4-99f9-4e40-abff-20f3aa4c174a service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:16.699734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58c6a4-99f9-4e40-abff-20f3aa4c174a service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:16.700015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58c6a4-99f9-4e40-abff-20f3aa4c174a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:16.701699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bf58c6a4-99f9-4e40-abff-20f3aa4c174a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:16.750200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:16.794407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58c6a4-99f9-4e40-abff-20f3aa4c174a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:16.796090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58c6a4-99f9-4e40-abff-20f3aa4c174a service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:16.814347 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 718/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:17.693400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cb094-36a0-4a41-b173-2250a4a75757 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:17.694016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cb094-36a0-4a41-b173-2250a4a75757 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:17.694016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cb094-36a0-4a41-b173-2250a4a75757 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:17.695359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:17.700160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d226193-93e5-49f4-8b4d-47eef17f183c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:17.700577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d226193-93e5-49f4-8b4d-47eef17f183c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:17.700711 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d226193-93e5-49f4-8b4d-47eef17f183c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:17.773903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4d226193-93e5-49f4-8b4d-47eef17f183c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:17.775117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4d226193-93e5-49f4-8b4d-47eef17f183c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:17.873289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:18.022735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cb094-36a0-4a41-b173-2250a4a75757 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:18.024255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-817cb094-36a0-4a41-b173-2250a4a75757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:18.081230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:18.184075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cb094-36a0-4a41-b173-2250a4a75757 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:18.296045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe98737-9729-4635-8536-e0163e256992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:18.296607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe98737-9729-4635-8536-e0163e256992 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:18.296889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe98737-9729-4635-8536-e0163e256992 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:18.297192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe98737-9729-4635-8536-e0163e256992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:18.298843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-efe98737-9729-4635-8536-e0163e256992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:18.305913 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:18.388773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe98737-9729-4635-8536-e0163e256992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:18.390442 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe98737-9729-4635-8536-e0163e256992 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:18.409569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 712/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:37:19.277418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610cba2-bffc-4e96-95f7-5d80f0faaca9 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:19.278001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610cba2-bffc-4e96-95f7-5d80f0faaca9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:19.278001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610cba2-bffc-4e96-95f7-5d80f0faaca9 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:19.279338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:19.284208 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9aafb-13a2-4886-a1ae-4c02e74f4bfe None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:19.284639 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9aafb-13a2-4886-a1ae-4c02e74f4bfe None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:19.284754 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9aafb-13a2-4886-a1ae-4c02e74f4bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:19.362562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-82d9aafb-13a2-4886-a1ae-4c02e74f4bfe None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:19.363783 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-82d9aafb-13a2-4886-a1ae-4c02e74f4bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:19.433452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:19.467849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:19.620542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610cba2-bffc-4e96-95f7-5d80f0faaca9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:19.622246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e610cba2-bffc-4e96-95f7-5d80f0faaca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:19.639798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:19.772736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610cba2-bffc-4e96-95f7-5d80f0faaca9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:19.859281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:19.885416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a2fb9-9d43-4576-9c31-7f3a157b4079 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:19.886064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a2fb9-9d43-4576-9c31-7f3a157b4079 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:19.886437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a2fb9-9d43-4576-9c31-7f3a157b4079 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:19.886620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a2fb9-9d43-4576-9c31-7f3a157b4079 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:19.888223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-810a2fb9-9d43-4576-9c31-7f3a157b4079 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:19.979062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a2fb9-9d43-4576-9c31-7f3a157b4079 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:19.980750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a2fb9-9d43-4576-9c31-7f3a157b4079 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:20.003369 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 721/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:20.867969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620672-b1b4-4ddd-af21-ff951769b423 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:20.868525 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620672-b1b4-4ddd-af21-ff951769b423 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:20.868525 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620672-b1b4-4ddd-af21-ff951769b423 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:20.869959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:20.875088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e582cfb-3801-4260-993f-47bde458e6e4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:20.875518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e582cfb-3801-4260-993f-47bde458e6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:20.875647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e582cfb-3801-4260-993f-47bde458e6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:20.950946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9e582cfb-3801-4260-993f-47bde458e6e4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:20.952162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9e582cfb-3801-4260-993f-47bde458e6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:20.993086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:21.056386 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:21.206724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620672-b1b4-4ddd-af21-ff951769b423 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:21.208774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-df620672-b1b4-4ddd-af21-ff951769b423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:21.353029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620672-b1b4-4ddd-af21-ff951769b423 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:21.421383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:21.480362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4b08f-2577-44c9-b391-ad21663bc673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:21.481021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b08f-2577-44c9-b391-ad21663bc673 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:21.481334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b08f-2577-44c9-b391-ad21663bc673 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:21.487939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b08f-2577-44c9-b391-ad21663bc673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:21.487939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-96a4b08f-2577-44c9-b391-ad21663bc673 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:21.573276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4b08f-2577-44c9-b391-ad21663bc673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:21.574878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4b08f-2577-44c9-b391-ad21663bc673 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:21.593645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 715/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:22.453957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529a1ea-01dd-436e-81d1-cd6ab5454c4e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:22.454558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529a1ea-01dd-436e-81d1-cd6ab5454c4e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:22.454558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529a1ea-01dd-436e-81d1-cd6ab5454c4e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:22.455946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:37:22.461536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb956b-4df5-4c36-a969-972708c0ad0a None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:22.461973 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb956b-4df5-4c36-a969-972708c0ad0a None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:22.462094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb956b-4df5-4c36-a969-972708c0ad0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:22.538814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ecbb956b-4df5-4c36-a969-972708c0ad0a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:22.540035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ecbb956b-4df5-4c36-a969-972708c0ad0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:22.542328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:22.633594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:22.782399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529a1ea-01dd-436e-81d1-cd6ab5454c4e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:22.783935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3529a1ea-01dd-436e-81d1-cd6ab5454c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:22.930793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529a1ea-01dd-436e-81d1-cd6ab5454c4e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:22.957026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:23.042586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d130702-8d80-4f6a-8b41-0d524acffe81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:23.043164 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d130702-8d80-4f6a-8b41-0d524acffe81 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:23.043450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d130702-8d80-4f6a-8b41-0d524acffe81 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:23.043725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d130702-8d80-4f6a-8b41-0d524acffe81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:23.045396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2d130702-8d80-4f6a-8b41-0d524acffe81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:23.140411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d130702-8d80-4f6a-8b41-0d524acffe81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:23.142051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d130702-8d80-4f6a-8b41-0d524acffe81 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:23.159762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 724/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:25.105742 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73c856-5e28-4d77-bb09-9224062902ff None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:25.106243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73c856-5e28-4d77-bb09-9224062902ff None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:25.106243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73c856-5e28-4d77-bb09-9224062902ff None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:25.107746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:25.112801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b71cd-9f2f-4dd6-9321-d14e605f44a5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:25.113241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b71cd-9f2f-4dd6-9321-d14e605f44a5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:25.113408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b71cd-9f2f-4dd6-9321-d14e605f44a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:25.155226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:25.181914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0a5b71cd-9f2f-4dd6-9321-d14e605f44a5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:25.183131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0a5b71cd-9f2f-4dd6-9321-d14e605f44a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:25.283491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:25.321997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73c856-5e28-4d77-bb09-9224062902ff None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:25.323555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5b73c856-5e28-4d77-bb09-9224062902ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:25.431547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73c856-5e28-4d77-bb09-9224062902ff None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:25.466255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:25.510968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6bc37-9227-49e2-8e17-851983651cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:25.511623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6bc37-9227-49e2-8e17-851983651cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:25.511907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6bc37-9227-49e2-8e17-851983651cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:25.512181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6bc37-9227-49e2-8e17-851983651cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:25.513893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e2f6bc37-9227-49e2-8e17-851983651cd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:25.622435 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6bc37-9227-49e2-8e17-851983651cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:25.623884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6bc37-9227-49e2-8e17-851983651cd9 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:25.642870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 718/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:27.844364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddf14c-4a36-4922-8f20-ff123bcfa0c7 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:27.844894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddf14c-4a36-4922-8f20-ff123bcfa0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:27.844986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddf14c-4a36-4922-8f20-ff123bcfa0c7 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:27.846357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:27.852472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403f7ae-035c-4303-9c28-f363bd75d7c8 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:27.852934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403f7ae-035c-4303-9c28-f363bd75d7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:27.853041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403f7ae-035c-4303-9c28-f363bd75d7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:27.917836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-7403f7ae-035c-4303-9c28-f363bd75d7c8 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:27.919036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7403f7ae-035c-4303-9c28-f363bd75d7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:27.931596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:28.016763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:28.054910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddf14c-4a36-4922-8f20-ff123bcfa0c7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:28.056462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-51ddf14c-4a36-4922-8f20-ff123bcfa0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:28.162811 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddf14c-4a36-4922-8f20-ff123bcfa0c7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:28.175745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:28.238940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c46c63-dce6-4db2-b9a0-f4f6663d7326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:28.239680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c46c63-dce6-4db2-b9a0-f4f6663d7326 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:28.240455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c46c63-dce6-4db2-b9a0-f4f6663d7326 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:28.240737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c46c63-dce6-4db2-b9a0-f4f6663d7326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:28.242427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e5c46c63-dce6-4db2-b9a0-f4f6663d7326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:28.349530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c46c63-dce6-4db2-b9a0-f4f6663d7326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:28.350894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c46c63-dce6-4db2-b9a0-f4f6663d7326 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:28.369047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 727/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:30.766970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd271-83e2-4efb-b7c7-5924bf5ea2c5 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:30.767587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd271-83e2-4efb-b7c7-5924bf5ea2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:30.767587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd271-83e2-4efb-b7c7-5924bf5ea2c5 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:30.768943 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:30.774690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078c17-ac27-40cc-878b-ca4d3a4c6385 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:30.778331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078c17-ac27-40cc-878b-ca4d3a4c6385 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:30.778331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078c17-ac27-40cc-878b-ca4d3a4c6385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:30.822973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:30.840856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-39078c17-ac27-40cc-878b-ca4d3a4c6385 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:30.842105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-39078c17-ac27-40cc-878b-ca4d3a4c6385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:30.941856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:30.981150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd271-83e2-4efb-b7c7-5924bf5ea2c5 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:30.982824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-624cd271-83e2-4efb-b7c7-5924bf5ea2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:31.085824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd271-83e2-4efb-b7c7-5924bf5ea2c5 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:31.125392 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:31.160516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783f1dd-e86e-4b3c-b95f-e7f420e73c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:31.161210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783f1dd-e86e-4b3c-b95f-e7f420e73c1b service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:31.161499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783f1dd-e86e-4b3c-b95f-e7f420e73c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:31.161777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783f1dd-e86e-4b3c-b95f-e7f420e73c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:31.163434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8783f1dd-e86e-4b3c-b95f-e7f420e73c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:31.272496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8783f1dd-e86e-4b3c-b95f-e7f420e73c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:31.273989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8783f1dd-e86e-4b3c-b95f-e7f420e73c1b service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:31.292514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 721/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:33.468610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c702e-46a0-4854-b83b-d2cdd2ce7e24 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:33.469177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c702e-46a0-4854-b83b-d2cdd2ce7e24 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:33.469177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c702e-46a0-4854-b83b-d2cdd2ce7e24 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:33.470616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:33.476055 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f78d0-11a9-453f-add2-2dd4cbe55eb2 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:33.476571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f78d0-11a9-453f-add2-2dd4cbe55eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:33.476691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f78d0-11a9-453f-add2-2dd4cbe55eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:33.543044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9c8f78d0-11a9-453f-add2-2dd4cbe55eb2 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:33.544265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9c8f78d0-11a9-453f-add2-2dd4cbe55eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:33.565271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:33.642664 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:33.681496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c702e-46a0-4854-b83b-d2cdd2ce7e24 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:33.683019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c80c702e-46a0-4854-b83b-d2cdd2ce7e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:33.919983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c702e-46a0-4854-b83b-d2cdd2ce7e24 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:33.939806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:33.993176 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3605ef-bed0-46b8-b772-aa9f5a82bd25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:33.993824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3605ef-bed0-46b8-b772-aa9f5a82bd25 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:33.994096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3605ef-bed0-46b8-b772-aa9f5a82bd25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:33.994361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3605ef-bed0-46b8-b772-aa9f5a82bd25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:33.995965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cb3605ef-bed0-46b8-b772-aa9f5a82bd25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:34.102822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3605ef-bed0-46b8-b772-aa9f5a82bd25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:34.104297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3605ef-bed0-46b8-b772-aa9f5a82bd25 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:34.122559 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 730/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:36.220039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aea136-8569-4fc5-81fc-769ac6cf6a67 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:36.220587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aea136-8569-4fc5-81fc-769ac6cf6a67 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:36.220587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aea136-8569-4fc5-81fc-769ac6cf6a67 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:36.221998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:36.229507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c9249-d9a2-457a-aa52-a5c91b6c5698 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:36.229971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c9249-d9a2-457a-aa52-a5c91b6c5698 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:36.230079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c9249-d9a2-457a-aa52-a5c91b6c5698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:36.292271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:36.295263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-476c9249-d9a2-457a-aa52-a5c91b6c5698 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:36.296471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-476c9249-d9a2-457a-aa52-a5c91b6c5698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:36.399371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:36.437150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aea136-8569-4fc5-81fc-769ac6cf6a67 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:36.438708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-17aea136-8569-4fc5-81fc-769ac6cf6a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:36.554615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aea136-8569-4fc5-81fc-769ac6cf6a67 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:36.603015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:36.627491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfbb62-6286-41e2-9870-07926f95d27f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:36.628138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfbb62-6286-41e2-9870-07926f95d27f service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:36.628416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfbb62-6286-41e2-9870-07926f95d27f service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:36.628691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfbb62-6286-41e2-9870-07926f95d27f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:36.630411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4fcfbb62-6286-41e2-9870-07926f95d27f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:36.736051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfbb62-6286-41e2-9870-07926f95d27f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:36.737525 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfbb62-6286-41e2-9870-07926f95d27f service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:36.755391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 724/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:38.887324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728b831-92ab-44d5-83c8-ccdb414abc2f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:38.887874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728b831-92ab-44d5-83c8-ccdb414abc2f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:38.887874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728b831-92ab-44d5-83c8-ccdb414abc2f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:38.889275 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:38.895381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867ced-8f5d-48c4-9f2c-0cab1990c1fc None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:38.895787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867ced-8f5d-48c4-9f2c-0cab1990c1fc None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:38.896058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867ced-8f5d-48c4-9f2c-0cab1990c1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:38.963299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-75867ced-8f5d-48c4-9f2c-0cab1990c1fc None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:38.964517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-75867ced-8f5d-48c4-9f2c-0cab1990c1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:38.993261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:39.060135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:39.097903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728b831-92ab-44d5-83c8-ccdb414abc2f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:39.100029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7728b831-92ab-44d5-83c8-ccdb414abc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:39.200896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728b831-92ab-44d5-83c8-ccdb414abc2f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:39.230342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:39.275523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d485eb5-c4ee-442e-850b-3de762c691cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:39.276160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d485eb5-c4ee-442e-850b-3de762c691cb service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:39.276436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d485eb5-c4ee-442e-850b-3de762c691cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:39.276705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d485eb5-c4ee-442e-850b-3de762c691cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:39.278721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3d485eb5-c4ee-442e-850b-3de762c691cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:39.392129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d485eb5-c4ee-442e-850b-3de762c691cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:39.393521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d485eb5-c4ee-442e-850b-3de762c691cb service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:39.412559 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 733/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:41.495301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa514f8-785b-4b70-a1f6-ff9e5f01928f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:41.495908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa514f8-785b-4b70-a1f6-ff9e5f01928f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:41.495908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa514f8-785b-4b70-a1f6-ff9e5f01928f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:41.497438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:41.502645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9513c54-4bf0-49d2-a13f-7cef2004b587 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:41.503112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9513c54-4bf0-49d2-a13f-7cef2004b587 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:41.503222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9513c54-4bf0-49d2-a13f-7cef2004b587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:41.566444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a9513c54-4bf0-49d2-a13f-7cef2004b587 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:41.568521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a9513c54-4bf0-49d2-a13f-7cef2004b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:41.576672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:41.680280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:41.718375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa514f8-785b-4b70-a1f6-ff9e5f01928f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:41.719952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8aa514f8-785b-4b70-a1f6-ff9e5f01928f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:41.836023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa514f8-785b-4b70-a1f6-ff9e5f01928f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:41.891447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:41.910014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83aa725-c587-4167-9dd2-5fb0ee08b88f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:41.910671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aa725-c587-4167-9dd2-5fb0ee08b88f service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:41.911000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aa725-c587-4167-9dd2-5fb0ee08b88f service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:41.911228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aa725-c587-4167-9dd2-5fb0ee08b88f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:41.912916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d83aa725-c587-4167-9dd2-5fb0ee08b88f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:42.020898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83aa725-c587-4167-9dd2-5fb0ee08b88f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:42.022390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83aa725-c587-4167-9dd2-5fb0ee08b88f service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:42.040981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 727/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:44.116357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fefddd-7038-45b2-85c6-50ce1043cb77 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:44.116960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fefddd-7038-45b2-85c6-50ce1043cb77 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:44.116960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fefddd-7038-45b2-85c6-50ce1043cb77 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:44.118316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:44.123550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5b206-f1ef-4248-a7db-bb9630af6d94 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:44.123958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5b206-f1ef-4248-a7db-bb9630af6d94 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:44.124097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5b206-f1ef-4248-a7db-bb9630af6d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:44.186989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-fab5b206-f1ef-4248-a7db-bb9630af6d94 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:44.188196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-fab5b206-f1ef-4248-a7db-bb9630af6d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:44.225105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:44.281764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:44.319693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fefddd-7038-45b2-85c6-50ce1043cb77 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:44.321233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-75fefddd-7038-45b2-85c6-50ce1043cb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:44.424813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fefddd-7038-45b2-85c6-50ce1043cb77 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:44.463692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:44.500156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31725b-340b-4cca-addf-8528f29d35b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:44.500823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31725b-340b-4cca-addf-8528f29d35b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:44.501464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31725b-340b-4cca-addf-8528f29d35b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:44.501898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31725b-340b-4cca-addf-8528f29d35b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:44.504015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6c31725b-340b-4cca-addf-8528f29d35b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:44.611764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31725b-340b-4cca-addf-8528f29d35b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:44.613236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31725b-340b-4cca-addf-8528f29d35b2 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:44.632592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 736/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:46.681919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeac7af-46a7-419c-a51b-c917447e5d57 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:46.682896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeac7af-46a7-419c-a51b-c917447e5d57 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:46.682896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeac7af-46a7-419c-a51b-c917447e5d57 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:46.684414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:46.689879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c98d2-129c-4a3b-990e-76593d91a1fb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:46.690352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c98d2-129c-4a3b-990e-76593d91a1fb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:46.690461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c98d2-129c-4a3b-990e-76593d91a1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:46.757019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ae1c98d2-129c-4a3b-990e-76593d91a1fb None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:46.758280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ae1c98d2-129c-4a3b-990e-76593d91a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:46.776726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:46.854817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:46.892006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeac7af-46a7-419c-a51b-c917447e5d57 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:46.893571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3eeac7af-46a7-419c-a51b-c917447e5d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:46.998712 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeac7af-46a7-419c-a51b-c917447e5d57 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:47.066578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:47.073241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fead80c-74ed-43dd-adbd-8d23a82c9ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:47.073925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead80c-74ed-43dd-adbd-8d23a82c9ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:47.074173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead80c-74ed-43dd-adbd-8d23a82c9ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:47.074443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fead80c-74ed-43dd-adbd-8d23a82c9ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:47.076087 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7fead80c-74ed-43dd-adbd-8d23a82c9ad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:47.184650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fead80c-74ed-43dd-adbd-8d23a82c9ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:47.186127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fead80c-74ed-43dd-adbd-8d23a82c9ad2 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:47.204035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 730/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:49.271773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0e5f-d55e-4dde-9f01-bb82c5036bf2 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:49.272371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0e5f-d55e-4dde-9f01-bb82c5036bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:49.272371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0e5f-d55e-4dde-9f01-bb82c5036bf2 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:49.273758 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:37:49.279004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96438bb2-3ce1-4e38-9e2e-b2ff2ee83b49 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:49.279414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96438bb2-3ce1-4e38-9e2e-b2ff2ee83b49 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:49.279564 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96438bb2-3ce1-4e38-9e2e-b2ff2ee83b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:49.344560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-96438bb2-3ce1-4e38-9e2e-b2ff2ee83b49 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:49.345821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-96438bb2-3ce1-4e38-9e2e-b2ff2ee83b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:49.397291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:49.446506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:49.482773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce0e5f-d55e-4dde-9f01-bb82c5036bf2 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:49.484695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-38ce0e5f-d55e-4dde-9f01-bb82c5036bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:49.589666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce0e5f-d55e-4dde-9f01-bb82c5036bf2 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:49.637113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:49.662513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c03f4c-6650-49a2-af4f-7112330a2967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:49.663153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c03f4c-6650-49a2-af4f-7112330a2967 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:49.663435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c03f4c-6650-49a2-af4f-7112330a2967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:49.663705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c03f4c-6650-49a2-af4f-7112330a2967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:49.665600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c0c03f4c-6650-49a2-af4f-7112330a2967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:49.772864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c03f4c-6650-49a2-af4f-7112330a2967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:49.776983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c03f4c-6650-49a2-af4f-7112330a2967 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:49.793024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 739/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:59.449594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=22 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:59.790692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=66 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:09.636875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b6d7f-c7bc-4aba-bd98-18c4d6240fb9 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:09.637621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b6d7f-c7bc-4aba-bd98-18c4d6240fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:09.637621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b6d7f-c7bc-4aba-bd98-18c4d6240fb9 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:09.638897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 732/1471] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:38:09.644606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6266f70-d39c-4dd9-b695-614cd9175d3d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:09.645002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6266f70-d39c-4dd9-b695-614cd9175d3d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:09.645172 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6266f70-d39c-4dd9-b695-614cd9175d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:09.708652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d6266f70-d39c-4dd9-b695-614cd9175d3d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:09.709888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d6266f70-d39c-4dd9-b695-614cd9175d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:09.807566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 740/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:09.893131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330bc03-91a9-4cab-bd23-3e15648a66bd None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:09.894003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330bc03-91a9-4cab-bd23-3e15648a66bd None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:09.894286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330bc03-91a9-4cab-bd23-3e15648a66bd None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:09.895676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 733/1473] 10.222.0.40 () {54 vars in 962 bytes} [Thu Jul 14 11:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:38:09.900489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73febf7f-e5bc-42f2-be08-dc58aaf0c2e9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:09.900806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73febf7f-e5bc-42f2-be08-dc58aaf0c2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:09.901077 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73febf7f-e5bc-42f2-be08-dc58aaf0c2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:09.975905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-73febf7f-e5bc-42f2-be08-dc58aaf0c2e9 None None] MFA Rules not processed for user `a1a7213cbf6448b7ba6570a52a4c7c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:09.976906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-73febf7f-e5bc-42f2-be08-dc58aaf0c2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:09.998910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:10.054876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 741/1474] 10.222.0.40 () {58 vars in 1059 bytes} [Thu Jul 14 11:38:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:10.062551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330bc03-91a9-4cab-bd23-3e15648a66bd None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:10.064107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e330bc03-91a9-4cab-bd23-3e15648a66bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:10.168254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330bc03-91a9-4cab-bd23-3e15648a66bd None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:10.241958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f91ac7-689f-4b84-aef0-b2fc0ff681a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1a7213cbf6448b7ba6570a52a4c7c0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:10.242599 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91ac7-689f-4b84-aef0-b2fc0ff681a2 service glance] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:10.242920 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91ac7-689f-4b84-aef0-b2fc0ff681a2 service glance] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:10.243188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91ac7-689f-4b84-aef0-b2fc0ff681a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:10.244842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-78f91ac7-689f-4b84-aef0-b2fc0ff681a2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:10.298734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:10.354610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f91ac7-689f-4b84-aef0-b2fc0ff681a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:10.356054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f91ac7-689f-4b84-aef0-b2fc0ff681a2 service glance] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:10.376962 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 734/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:12.519289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec951d48-0f5b-456a-93ac-572e6d6c14ff None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.519862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec951d48-0f5b-456a-93ac-572e6d6c14ff None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.519862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec951d48-0f5b-456a-93ac-572e6d6c14ff None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.521238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 742/1476] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:38:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:38:12.526626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a199ce-d2b8-4102-8737-8ffa3fe9e298 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.527057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a199ce-d2b8-4102-8737-8ffa3fe9e298 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.527210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a199ce-d2b8-4102-8737-8ffa3fe9e298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.590950 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-54a199ce-d2b8-4102-8737-8ffa3fe9e298 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:12.592193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-54a199ce-d2b8-4102-8737-8ffa3fe9e298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:12.688728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 735/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:12.712738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec951d48-0f5b-456a-93ac-572e6d6c14ff None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:12.714288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ec951d48-0f5b-456a-93ac-572e6d6c14ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:12.845586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec951d48-0f5b-456a-93ac-572e6d6c14ff None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:12.896043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:12.941376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a6901-5701-4dae-8cf2-b474ac9bbbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:12.942023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a6901-5701-4dae-8cf2-b474ac9bbbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.942317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a6901-5701-4dae-8cf2-b474ac9bbbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.942574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a6901-5701-4dae-8cf2-b474ac9bbbd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.943297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a6901-5701-4dae-8cf2-b474ac9bbbd5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:12.945191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-4c6a6901-5701-4dae-8cf2-b474ac9bbbd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:38:12.945191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Jul 14 11:38:12.945191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: Jul 14 11:38:12.946322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a6901-5701-4dae-8cf2-b474ac9bbbd5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:12.965552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 743/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jul 14 11:38:12 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:15.047764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636460a3-a492-4736-b2e8-c7718eb57642 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:15.048294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636460a3-a492-4736-b2e8-c7718eb57642 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:15.048294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636460a3-a492-4736-b2e8-c7718eb57642 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:15.050199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 736/1479] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:38:15.055376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2581eb4-3ec2-4c47-8523-cd9a951b2d86 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:15.055827 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2581eb4-3ec2-4c47-8523-cd9a951b2d86 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:15.055945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2581eb4-3ec2-4c47-8523-cd9a951b2d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:15.119754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b2581eb4-3ec2-4c47-8523-cd9a951b2d86 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:15.120963 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b2581eb4-3ec2-4c47-8523-cd9a951b2d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:15.214767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 744/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:38:15.258219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636460a3-a492-4736-b2e8-c7718eb57642 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:15.259777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-636460a3-a492-4736-b2e8-c7718eb57642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:15.269985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:15.407474 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636460a3-a492-4736-b2e8-c7718eb57642 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:15.489636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:15.521887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0b648-eeeb-42f0-b0bb-52f599556466 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:15.522790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0b648-eeeb-42f0-b0bb-52f599556466 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:15.523076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0b648-eeeb-42f0-b0bb-52f599556466 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:15.523552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0b648-eeeb-42f0-b0bb-52f599556466 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:15.525656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-79e0b648-eeeb-42f0-b0bb-52f599556466 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:15.612680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0b648-eeeb-42f0-b0bb-52f599556466 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:15.614325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0b648-eeeb-42f0-b0bb-52f599556466 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:15.632436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 737/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:17.863399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32dd23-5492-4eda-9668-6156e688e65a None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:17.863992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32dd23-5492-4eda-9668-6156e688e65a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:17.863992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32dd23-5492-4eda-9668-6156e688e65a None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:17.865436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 745/1482] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:38:17.871004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a3e77-5887-4e50-a5cd-2d6c05134fa4 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:17.871475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a3e77-5887-4e50-a5cd-2d6c05134fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:17.871586 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a3e77-5887-4e50-a5cd-2d6c05134fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:17.936533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-490a3e77-5887-4e50-a5cd-2d6c05134fa4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:17.937792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-490a3e77-5887-4e50-a5cd-2d6c05134fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:17.985414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:18.035046 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 738/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:38:18.070995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32dd23-5492-4eda-9668-6156e688e65a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:18.072547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3d32dd23-5492-4eda-9668-6156e688e65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:18.096209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:18.176281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32dd23-5492-4eda-9668-6156e688e65a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:18.252775 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ad36a-15f8-4773-b9b0-b4b84cc68a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:18.253456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ad36a-15f8-4773-b9b0-b4b84cc68a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:18.253740 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ad36a-15f8-4773-b9b0-b4b84cc68a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:18.254013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ad36a-15f8-4773-b9b0-b4b84cc68a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:18.255676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-649ad36a-15f8-4773-b9b0-b4b84cc68a25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:18.331222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:18.366249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ad36a-15f8-4773-b9b0-b4b84cc68a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:18.367734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ad36a-15f8-4773-b9b0-b4b84cc68a25 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:18.386453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 746/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:20.483993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cf3f9-1ac5-4f8f-9dd2-14a550b70c77 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:20.484551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cf3f9-1ac5-4f8f-9dd2-14a550b70c77 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:20.484551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cf3f9-1ac5-4f8f-9dd2-14a550b70c77 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:20.485930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 739/1485] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:38:20.492007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d00171-53d9-4984-bb65-b9466325b0cc None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:20.492453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d00171-53d9-4984-bb65-b9466325b0cc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:20.492565 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d00171-53d9-4984-bb65-b9466325b0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:20.556138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e6d00171-53d9-4984-bb65-b9466325b0cc None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:20.557370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e6d00171-53d9-4984-bb65-b9466325b0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:20.651684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 747/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:20.689999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cf3f9-1ac5-4f8f-9dd2-14a550b70c77 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:20.691603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9e0cf3f9-1ac5-4f8f-9dd2-14a550b70c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:20.794976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cf3f9-1ac5-4f8f-9dd2-14a550b70c77 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:20.867195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:20.869568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc7b1-bad9-4050-86c3-3ab095e09298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:20.870223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc7b1-bad9-4050-86c3-3ab095e09298 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:20.870505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc7b1-bad9-4050-86c3-3ab095e09298 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:20.870777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc7b1-bad9-4050-86c3-3ab095e09298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:20.872460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-39fdc7b1-bad9-4050-86c3-3ab095e09298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:20.982875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc7b1-bad9-4050-86c3-3ab095e09298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:20.984230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc7b1-bad9-4050-86c3-3ab095e09298 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:21.001808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 740/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:23.143444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb35ad3-8b0d-4396-9efa-de8f3e692408 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:23.143977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb35ad3-8b0d-4396-9efa-de8f3e692408 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:23.143977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb35ad3-8b0d-4396-9efa-de8f3e692408 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:23.145320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 748/1488] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:38:23.150640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84b620-7441-47ae-ab37-58fd25f7c238 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:23.151077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84b620-7441-47ae-ab37-58fd25f7c238 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:23.151196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84b620-7441-47ae-ab37-58fd25f7c238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:23.218151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-5d84b620-7441-47ae-ab37-58fd25f7c238 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:23.219404 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5d84b620-7441-47ae-ab37-58fd25f7c238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:23.272604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:23.317274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 741/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:23.357141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb35ad3-8b0d-4396-9efa-de8f3e692408 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:23.358688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7fb35ad3-8b0d-4396-9efa-de8f3e692408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:23.395564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:23.463773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb35ad3-8b0d-4396-9efa-de8f3e692408 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:23.546047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aff580-d9e6-46ab-a019-0f3eb932c37d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:23.546694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aff580-d9e6-46ab-a019-0f3eb932c37d service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:23.546972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aff580-d9e6-46ab-a019-0f3eb932c37d service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:23.547307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aff580-d9e6-46ab-a019-0f3eb932c37d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:23.549036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d7aff580-d9e6-46ab-a019-0f3eb932c37d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:23.630512 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:23.658641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aff580-d9e6-46ab-a019-0f3eb932c37d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:23.660492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aff580-d9e6-46ab-a019-0f3eb932c37d service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:23.678694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 749/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:26.215561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c1db5-017d-4b94-88ac-b7216287b6f9 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:26.216125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c1db5-017d-4b94-88ac-b7216287b6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:26.216125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c1db5-017d-4b94-88ac-b7216287b6f9 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:26.217538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 742/1491] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jul 14 11:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:38:26.223198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee255c-488a-4a63-b027-55fbd4698927 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:26.223629 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee255c-488a-4a63-b027-55fbd4698927 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:26.223782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee255c-488a-4a63-b027-55fbd4698927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:26.290563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9aee255c-488a-4a63-b027-55fbd4698927 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:26.291778 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9aee255c-488a-4a63-b027-55fbd4698927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:26.389127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 750/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jul 14 11:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:26.428535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c1db5-017d-4b94-88ac-b7216287b6f9 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:26.430202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-094c1db5-017d-4b94-88ac-b7216287b6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:26.531825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c1db5-017d-4b94-88ac-b7216287b6f9 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:26.606786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b1eb1-5baa-4e91-9dbd-b3d0f6bb53e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:26.607432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b1eb1-5baa-4e91-9dbd-b3d0f6bb53e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:26.607713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b1eb1-5baa-4e91-9dbd-b3d0f6bb53e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:26.607987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b1eb1-5baa-4e91-9dbd-b3d0f6bb53e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:26.609719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7c7b1eb1-5baa-4e91-9dbd-b3d0f6bb53e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:26.618808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:26.717304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b1eb1-5baa-4e91-9dbd-b3d0f6bb53e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:26.718750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b1eb1-5baa-4e91-9dbd-b3d0f6bb53e5 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:26.736981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 743/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:36.387277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=82 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:36.734750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=50 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:03.983812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c73a8b-5a29-49cf-8f4a-44544bb8325a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:03.984426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c73a8b-5a29-49cf-8f4a-44544bb8325a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:03.984426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c73a8b-5a29-49cf-8f4a-44544bb8325a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:04.057712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-28c73a8b-5a29-49cf-8f4a-44544bb8325a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:40:04.059471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-28c73a8b-5a29-49cf-8f4a-44544bb8325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:04.158223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 751/1494] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:40:04.166547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d925-bb59-4f16-9315-f0215e6a8c2c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:04.167045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d925-bb59-4f16-9315-f0215e6a8c2c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:04.167146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d925-bb59-4f16-9315-f0215e6a8c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:04.241452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ee27d925-bb59-4f16-9315-f0215e6a8c2c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:40:04.242674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ee27d925-bb59-4f16-9315-f0215e6a8c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:04.341854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 744/1495] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:40:04.346448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c73a8b-5a29-49cf-8f4a-44544bb8325a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:04.347741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-28c73a8b-5a29-49cf-8f4a-44544bb8325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:04.359624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c73a8b-5a29-49cf-8f4a-44544bb8325a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:04.423875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:04.458236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:04.458912 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:04.459388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:04.459711 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:04.460650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:40:04.461014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:04.462391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:40:04.462391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Jul 14 11:40:04.462391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: Jul 14 11:40:04.463277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:04.475359 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 752/1496] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:40:04.484177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d925-bb59-4f16-9315-f0215e6a8c2c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:04.493581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d925-bb59-4f16-9315-f0215e6a8c2c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:04.553310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:04.585710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc58ee-981e-4a3b-9c2c-f55e33c29fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:04.586368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc58ee-981e-4a3b-9c2c-f55e33c29fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:04.586647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc58ee-981e-4a3b-9c2c-f55e33c29fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:04.586917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc58ee-981e-4a3b-9c2c-f55e33c29fa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:04.587599 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc58ee-981e-4a3b-9c2c-f55e33c29fa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:04.589395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: WARNING py.warnings [None req-ebcc58ee-981e-4a3b-9c2c-f55e33c29fa1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:40:04.589395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: warnings.warn(msg) Jul 14 11:40:04.589395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: Jul 14 11:40:04.591127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc58ee-981e-4a3b-9c2c-f55e33c29fa1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:04.757895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 745/1497] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:40:04 2022] POST /identity/v3/projects => generated 379 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:40:04.762575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:04.764313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:04.797772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84c89-9c22-4a98-a9ab-0024c2ec8f92 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:04.893122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bda01-1125-4551-815a-614c9a119117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:04.897177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bda01-1125-4551-815a-614c9a119117 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:04.897177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bda01-1125-4551-815a-614c9a119117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:04.897177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bda01-1125-4551-815a-614c9a119117 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:04.897177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bda01-1125-4551-815a-614c9a119117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:04.897177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-3b9bda01-1125-4551-815a-614c9a119117 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:40:04.897177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Jul 14 11:40:04.897177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: Jul 14 11:40:04.897878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bda01-1125-4551-815a-614c9a119117 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:05.033462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 753/1498] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:40:04 2022] POST /identity/v3/users => generated 290 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:40:05.047238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc58ee-981e-4a3b-9c2c-f55e33c29fa1 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:05.048738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ebcc58ee-981e-4a3b-9c2c-f55e33c29fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:05.110689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc58ee-981e-4a3b-9c2c-f55e33c29fa1 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:05.192195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:05.202678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7449-4abe-4f41-a06c-e29fb757fc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:05.206004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7449-4abe-4f41-a06c-e29fb757fc92 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:05.206004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7449-4abe-4f41-a06c-e29fb757fc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:05.206004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7449-4abe-4f41-a06c-e29fb757fc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:05.206004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7449-4abe-4f41-a06c-e29fb757fc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:05.206004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: WARNING py.warnings [None req-de0e7449-4abe-4f41-a06c-e29fb757fc92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:40:05.206004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: warnings.warn(msg) Jul 14 11:40:05.206004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: Jul 14 11:40:05.206682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7449-4abe-4f41-a06c-e29fb757fc92 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:05.220054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 746/1499] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:40:05.224276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bda01-1125-4551-815a-614c9a119117 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:05.225845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3b9bda01-1125-4551-815a-614c9a119117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:05.237374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bda01-1125-4551-815a-614c9a119117 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:05.263298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:05.333093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b0147-ae7e-4b07-9bf7-252a03dea628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:05.333764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b0147-ae7e-4b07-9bf7-252a03dea628 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:05.334048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b0147-ae7e-4b07-9bf7-252a03dea628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:05.334322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b0147-ae7e-4b07-9bf7-252a03dea628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:05.335062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b0147-ae7e-4b07-9bf7-252a03dea628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:05.336085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-465b0147-ae7e-4b07-9bf7-252a03dea628 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:40:05.336085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Jul 14 11:40:05.336085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: Jul 14 11:40:05.337015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b0147-ae7e-4b07-9bf7-252a03dea628 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:05.350653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 754/1500] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:40:05.354907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7449-4abe-4f41-a06c-e29fb757fc92 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:05.356521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-de0e7449-4abe-4f41-a06c-e29fb757fc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:05.393124 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7449-4abe-4f41-a06c-e29fb757fc92 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:05.494117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092d5b3-4c24-488b-9bf0-ae9426124872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:05.495026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092d5b3-4c24-488b-9bf0-ae9426124872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:05.495306 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092d5b3-4c24-488b-9bf0-ae9426124872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:05.495578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092d5b3-4c24-488b-9bf0-ae9426124872 admin admin] PATH_INFO: `/v3/projects/6c2ff6fc7bf44bafa1d289b2afd408c3/users/277d632577b040488bfbfa4964504d3d/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:05.518365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092d5b3-4c24-488b-9bf0-ae9426124872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2ff6fc7bf44bafa1d289b2afd408c3, user_id=277d632577b040488bfbfa4964504d3d, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:05.519707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: WARNING py.warnings [None req-c092d5b3-4c24-488b-9bf0-ae9426124872 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:40:05.519707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: warnings.warn(msg) Jul 14 11:40:05.519707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: Jul 14 11:40:05.521661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092d5b3-4c24-488b-9bf0-ae9426124872 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:05.660285 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 747/1501] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:40:05 2022] PUT /identity/v3/projects/6c2ff6fc7bf44bafa1d289b2afd408c3/users/277d632577b040488bfbfa4964504d3d/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:40:05.673239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5982-3d38-4017-9f40-e5212b87f367 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:05.673536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5982-3d38-4017-9f40-e5212b87f367 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:05.673814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5982-3d38-4017-9f40-e5212b87f367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:05.728879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b18e5982-3d38-4017-9f40-e5212b87f367 None None] MFA Rules not processed for user `277d632577b040488bfbfa4964504d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:40:05.730109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b18e5982-3d38-4017-9f40-e5212b87f367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:05.809892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 755/1502] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:40:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:40:05.822957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89b1ee-b7b7-40bb-ba57-c841ec88a3d6 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:05.826430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89b1ee-b7b7-40bb-ba57-c841ec88a3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:05.826430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89b1ee-b7b7-40bb-ba57-c841ec88a3d6 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:05.826430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 748/1503] 10.222.0.40 () {54 vars in 963 bytes} [Thu Jul 14 11:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:40:05.829603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea834ae7-6ee6-4a4d-bca2-d5739a8224bd None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:05.829896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea834ae7-6ee6-4a4d-bca2-d5739a8224bd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:05.830168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea834ae7-6ee6-4a4d-bca2-d5739a8224bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:05.859500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:05.904899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ea834ae7-6ee6-4a4d-bca2-d5739a8224bd None None] MFA Rules not processed for user `f12e348368ac42c696623b4a8f781977`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:40:05.906119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ea834ae7-6ee6-4a4d-bca2-d5739a8224bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:05.979215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 756/1504] 10.222.0.40 () {58 vars in 1060 bytes} [Thu Jul 14 11:40:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:40:05.987284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89b1ee-b7b7-40bb-ba57-c841ec88a3d6 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:05.988912 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8d89b1ee-b7b7-40bb-ba57-c841ec88a3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:06.050642 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:06.103630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89b1ee-b7b7-40bb-ba57-c841ec88a3d6 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:06.178788 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84734f82-618e-4374-9b53-0146cb5dc351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:06.179439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84734f82-618e-4374-9b53-0146cb5dc351 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:06.179716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84734f82-618e-4374-9b53-0146cb5dc351 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:06.179989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84734f82-618e-4374-9b53-0146cb5dc351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:06.181669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-84734f82-618e-4374-9b53-0146cb5dc351 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:06.279772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84734f82-618e-4374-9b53-0146cb5dc351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:06.281248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84734f82-618e-4374-9b53-0146cb5dc351 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:06.284976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:06.299434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 749/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:40:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:40:06.341880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89cd08-c0bb-4330-a018-792f017c566f None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:06.342306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89cd08-c0bb-4330-a018-792f017c566f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:06.342439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89cd08-c0bb-4330-a018-792f017c566f None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:06.343733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 757/1506] 10.222.0.40 () {52 vars in 829 bytes} [Thu Jul 14 11:40:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:40:06.347220 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84734f82-618e-4374-9b53-0146cb5dc351 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:06.348419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-84734f82-618e-4374-9b53-0146cb5dc351 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:06.446681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84734f82-618e-4374-9b53-0146cb5dc351 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:06.538797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:06.539570 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:06.539848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:06.540116 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] PATH_INFO: `/v3/users/277d632577b040488bfbfa4964504d3d` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:06.552015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=277d632577b040488bfbfa4964504d3d)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:06.553047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: WARNING py.warnings [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:40:06.553047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: warnings.warn(msg) Jul 14 11:40:06.553047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: Jul 14 11:40:06.554268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:06.563698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:06.763380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '277d632577b040488bfbfa4964504d3d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:40:06.763832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '277d632577b040488bfbfa4964504d3d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:40:06.781708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '277d632577b040488bfbfa4964504d3d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:40:06.794108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '277d632577b040488bfbfa4964504d3d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:40:06.853630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '277d632577b040488bfbfa4964504d3d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:40:06.859741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 750/1507] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:40:06 2022] DELETE /identity/v3/users/277d632577b040488bfbfa4964504d3d => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:40:06.864164 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba43c4-f8eb-4e82-b87a-1dd922a02cb6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:06.864459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba43c4-f8eb-4e82-b87a-1dd922a02cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:06.864742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba43c4-f8eb-4e82-b87a-1dd922a02cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:06.920723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-75ba43c4-f8eb-4e82-b87a-1dd922a02cb6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:40:06.921972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-75ba43c4-f8eb-4e82-b87a-1dd922a02cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:07.003918 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:07.016075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 758/1508] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:40:07.025907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:07.027144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:07.127501 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797347b-916a-4733-bc8b-aba315e870d4 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:07.183329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:07.202629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403a2e5-c1e7-413d-a0d9-fa38366dfa40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:07.203331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a2e5-c1e7-413d-a0d9-fa38366dfa40 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:07.203615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a2e5-c1e7-413d-a0d9-fa38366dfa40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:07.203897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a2e5-c1e7-413d-a0d9-fa38366dfa40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:07.205589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2403a2e5-c1e7-413d-a0d9-fa38366dfa40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:07.317090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403a2e5-c1e7-413d-a0d9-fa38366dfa40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:07.318619 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403a2e5-c1e7-413d-a0d9-fa38366dfa40 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:07.337175 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 751/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:40:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:40:07.775509 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba43c4-f8eb-4e82-b87a-1dd922a02cb6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:07.779008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-75ba43c4-f8eb-4e82-b87a-1dd922a02cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:07.788654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba43c4-f8eb-4e82-b87a-1dd922a02cb6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:07.892505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:07.893428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:07.893705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:07.893982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] PATH_INFO: `/v3/projects/6c2ff6fc7bf44bafa1d289b2afd408c3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:07.904526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2ff6fc7bf44bafa1d289b2afd408c3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:07.905586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:40:07.905586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Jul 14 11:40:07.905586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: Jul 14 11:40:07.906783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:08.012758 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2ff6fc7bf44bafa1d289b2afd408c3', 'request_id': 'req-c08b5ce2-f8ca-4472-b5e2-c17243de9def'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:40:08.119411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2ff6fc7bf44bafa1d289b2afd408c3', 'request_id': 'req-c08b5ce2-f8ca-4472-b5e2-c17243de9def'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:40:08.126859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2ff6fc7bf44bafa1d289b2afd408c3', 'request_id': 'req-c08b5ce2-f8ca-4472-b5e2-c17243de9def'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:40:08.163464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] The token cache is being invalidate because project 6c2ff6fc7bf44bafa1d289b2afd408c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:40:08.163752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:40:08.164298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c08b5ce2-f8ca-4472-b5e2-c17243de9def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:40:08.165699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 759/1510] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:40:07 2022] DELETE /identity/v3/projects/6c2ff6fc7bf44bafa1d289b2afd408c3 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:40:17.334696 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=62 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:18.161235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:41:51.892275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcac648-f669-4e28-8ee7-0028e2c2701b None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:51.892927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcac648-f669-4e28-8ee7-0028e2c2701b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:51.892927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcac648-f669-4e28-8ee7-0028e2c2701b None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:51.894241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:41:51.899596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f7603-a3d7-409f-9b2e-5253a19c1bf4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:51.900003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f7603-a3d7-409f-9b2e-5253a19c1bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:51.900140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f7603-a3d7-409f-9b2e-5253a19c1bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:51.981459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-698f7603-a3d7-409f-9b2e-5253a19c1bf4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:41:51.982680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-698f7603-a3d7-409f-9b2e-5253a19c1bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:41:52.087139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:41:52.241726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcac648-f669-4e28-8ee7-0028e2c2701b None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:41:52.246219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-afcac648-f669-4e28-8ee7-0028e2c2701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:41:52.365277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcac648-f669-4e28-8ee7-0028e2c2701b None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:41:52.373484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055d1b3-0a2a-4df7-8c4b-f8a4678b265b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:41:52.374147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d1b3-0a2a-4df7-8c4b-f8a4678b265b service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:52.374425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d1b3-0a2a-4df7-8c4b-f8a4678b265b service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:52.374693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d1b3-0a2a-4df7-8c4b-f8a4678b265b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:52.383460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055d1b3-0a2a-4df7-8c4b-f8a4678b265b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:41:52.385060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055d1b3-0a2a-4df7-8c4b-f8a4678b265b service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:41:52.394920 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 753/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:41:53.292533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953e78b-5bf2-42d1-9f5f-7073970b8a81 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:53.293297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953e78b-5bf2-42d1-9f5f-7073970b8a81 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:53.293297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953e78b-5bf2-42d1-9f5f-7073970b8a81 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:53.294547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:41:53.299390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f6bea-8167-4f40-ba37-49d8e057b57f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:53.299978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f6bea-8167-4f40-ba37-49d8e057b57f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:53.299978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f6bea-8167-4f40-ba37-49d8e057b57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:53.319611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-3b9f6bea-8167-4f40-ba37-49d8e057b57f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:41:53.320823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3b9f6bea-8167-4f40-ba37-49d8e057b57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:41:53.329070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:41:53.484170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953e78b-5bf2-42d1-9f5f-7073970b8a81 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:41:53.501922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953e78b-5bf2-42d1-9f5f-7073970b8a81 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:41:53.508588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e95c3-54c8-42d9-8459-3a736cf21b7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:41:53.509268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e95c3-54c8-42d9-8459-3a736cf21b7b service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:53.509545 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e95c3-54c8-42d9-8459-3a736cf21b7b service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:53.509817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e95c3-54c8-42d9-8459-3a736cf21b7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:53.518906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e95c3-54c8-42d9-8459-3a736cf21b7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:41:53.529190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e95c3-54c8-42d9-8459-3a736cf21b7b service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:41:53.533104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 762/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:41:54.039132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3e21e-e4a5-4d42-a85b-d09544c514cf None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:54.039431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3e21e-e4a5-4d42-a85b-d09544c514cf None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:54.039734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3e21e-e4a5-4d42-a85b-d09544c514cf None None] PATH_INFO: `/v3` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:54.041047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 755/1517] 10.222.0.55 () {54 vars in 925 bytes} [Thu Jul 14 11:41:54 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:41:54.047060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfa819-b8e2-43bd-9e73-5d698dfabfac None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:54.047354 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfa819-b8e2-43bd-9e73-5d698dfabfac None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:54.047634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfa819-b8e2-43bd-9e73-5d698dfabfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:54.105848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-21bfa819-b8e2-43bd-9e73-5d698dfabfac None None] MFA Rules not processed for user `fa879d581c6d44e18a8703e66ffbd549`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:41:54.107089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-21bfa819-b8e2-43bd-9e73-5d698dfabfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:41:54.124126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:41:54.177743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 763/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Jul 14 11:41:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:41:58.229279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ef3d4-586a-4c0b-9a60-a2648896ad6a None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:58.229827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ef3d4-586a-4c0b-9a60-a2648896ad6a None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:58.229827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ef3d4-586a-4c0b-9a60-a2648896ad6a None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:58.231357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 756/1519] 10.222.0.40 () {54 vars in 960 bytes} [Thu Jul 14 11:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:41:58.236907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c0a64-c373-4166-a8ef-33a202c05fcf None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:58.237418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c0a64-c373-4166-a8ef-33a202c05fcf None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:58.237584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c0a64-c373-4166-a8ef-33a202c05fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:58.259288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0c8c0a64-c373-4166-a8ef-33a202c05fcf None None] MFA Rules not processed for user `fa879d581c6d44e18a8703e66ffbd549`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:41:58.261016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0c8c0a64-c373-4166-a8ef-33a202c05fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:41:58.269529 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 764/1520] 10.222.0.40 () {58 vars in 1057 bytes} [Thu Jul 14 11:41:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:41:58.277264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ef3d4-586a-4c0b-9a60-a2648896ad6a None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:41:58.290678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ef3d4-586a-4c0b-9a60-a2648896ad6a None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:41:58.295512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c29a3-ca49-4fca-8e49-817fbc733d23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:41:58.296142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c29a3-ca49-4fca-8e49-817fbc733d23 service placement] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:41:58.296420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c29a3-ca49-4fca-8e49-817fbc733d23 service placement] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:41:58.296695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c29a3-ca49-4fca-8e49-817fbc733d23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:41:58.305330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c29a3-ca49-4fca-8e49-817fbc733d23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:41:58.306908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c29a3-ca49-4fca-8e49-817fbc733d23 service placement] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:41:58.316468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 757/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:41:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:42:04.781309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfe36e-3c02-4b39-9dd9-cdd09513f778 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:04.781940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfe36e-3c02-4b39-9dd9-cdd09513f778 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:04.781940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfe36e-3c02-4b39-9dd9-cdd09513f778 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:04.783432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:42:04.788290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ef9e5-0489-476e-97b2-4331e00af966 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:04.788760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ef9e5-0489-476e-97b2-4331e00af966 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:04.788870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ef9e5-0489-476e-97b2-4331e00af966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:04.807376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-d44ef9e5-0489-476e-97b2-4331e00af966 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:42:04.808555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d44ef9e5-0489-476e-97b2-4331e00af966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:42:04.816692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:42:04.972815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfe36e-3c02-4b39-9dd9-cdd09513f778 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:42:04.987561 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfe36e-3c02-4b39-9dd9-cdd09513f778 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:42:04.993653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e5d4d-aa8d-4c3c-9369-b5c1414f3c77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:42:04.994318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e5d4d-aa8d-4c3c-9369-b5c1414f3c77 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:04.994606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e5d4d-aa8d-4c3c-9369-b5c1414f3c77 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:04.994886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e5d4d-aa8d-4c3c-9369-b5c1414f3c77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:05.004375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e5d4d-aa8d-4c3c-9369-b5c1414f3c77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:42:05.006083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e5d4d-aa8d-4c3c-9369-b5c1414f3c77 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:42:05.018556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 766/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:42:05.832806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa14131-d73e-4713-9bda-cba9d615db61 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:05.833370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa14131-d73e-4713-9bda-cba9d615db61 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:05.833370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa14131-d73e-4713-9bda-cba9d615db61 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:05.834741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:42:05.840044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a435948-6ce1-4063-b137-799dfef20a40 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:05.840464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a435948-6ce1-4063-b137-799dfef20a40 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:05.840609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a435948-6ce1-4063-b137-799dfef20a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:05.859028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8a435948-6ce1-4063-b137-799dfef20a40 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:42:05.860231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8a435948-6ce1-4063-b137-799dfef20a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:42:05.868260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:42:06.021685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa14131-d73e-4713-9bda-cba9d615db61 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:42:06.036659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa14131-d73e-4713-9bda-cba9d615db61 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:42:06.042158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a285b-6f6f-4ee8-afc5-864f22ee2bca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:42:06.042782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a285b-6f6f-4ee8-afc5-864f22ee2bca service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:06.043060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a285b-6f6f-4ee8-afc5-864f22ee2bca service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:06.043329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a285b-6f6f-4ee8-afc5-864f22ee2bca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:06.052167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a285b-6f6f-4ee8-afc5-864f22ee2bca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:42:06.053812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a285b-6f6f-4ee8-afc5-864f22ee2bca service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:42:06.063272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 760/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:42:10.165589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8696873-c3f4-467a-9369-23f0f11e6b7e None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:10.166121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8696873-c3f4-467a-9369-23f0f11e6b7e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:10.166121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8696873-c3f4-467a-9369-23f0f11e6b7e None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:10.167431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:42:10.172526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f609d-f034-4332-98ae-15b1547ead27 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:10.172975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f609d-f034-4332-98ae-15b1547ead27 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:10.173084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f609d-f034-4332-98ae-15b1547ead27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:10.191180 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-f72f609d-f034-4332-98ae-15b1547ead27 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:42:10.192366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f72f609d-f034-4332-98ae-15b1547ead27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:42:10.200859 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:42:10.276689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8696873-c3f4-467a-9369-23f0f11e6b7e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:42:10.278669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c8696873-c3f4-467a-9369-23f0f11e6b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:42:10.353903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8696873-c3f4-467a-9369-23f0f11e6b7e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:42:10.357416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea648b1c-170d-4165-96c0-3180efc45133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:42:10.358051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea648b1c-170d-4165-96c0-3180efc45133 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:10.358332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea648b1c-170d-4165-96c0-3180efc45133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:10.358602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea648b1c-170d-4165-96c0-3180efc45133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:10.367947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea648b1c-170d-4165-96c0-3180efc45133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:42:10.369408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea648b1c-170d-4165-96c0-3180efc45133 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:42:10.379126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 769/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:42:12.415283 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f639b-c271-4384-a1a6-3d76eef9a39f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:12.415855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f639b-c271-4384-a1a6-3d76eef9a39f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:12.415855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f639b-c271-4384-a1a6-3d76eef9a39f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:12.417456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:42:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:42:12.422671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa44e-a214-4d76-8029-ed0a65a85c3e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:12.423139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa44e-a214-4d76-8029-ed0a65a85c3e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:12.423244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa44e-a214-4d76-8029-ed0a65a85c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:12.443086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-73efa44e-a214-4d76-8029-ed0a65a85c3e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:42:12.444288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-73efa44e-a214-4d76-8029-ed0a65a85c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:42:12.452999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:42:12.489895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f639b-c271-4384-a1a6-3d76eef9a39f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:42:12.492082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c50f639b-c271-4384-a1a6-3d76eef9a39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:42:12.506082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f639b-c271-4384-a1a6-3d76eef9a39f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:42:12.509869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8d705-3d70-4ac7-8935-3c38fb8c87b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:42:12.510510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8d705-3d70-4ac7-8935-3c38fb8c87b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:12.510823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8d705-3d70-4ac7-8935-3c38fb8c87b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:12.511098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8d705-3d70-4ac7-8935-3c38fb8c87b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:12.520275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8d705-3d70-4ac7-8935-3c38fb8c87b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:42:12.521762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8d705-3d70-4ac7-8935-3c38fb8c87b1 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:42:12.532160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 763/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:42:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:42:16.363167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb5f88-caab-49d8-b408-57890198a964 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:16.363737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb5f88-caab-49d8-b408-57890198a964 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:16.363737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb5f88-caab-49d8-b408-57890198a964 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:16.367286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:42:16.370999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de3bbe-9283-46c1-9ae7-40391021c17a None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:16.371470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de3bbe-9283-46c1-9ae7-40391021c17a None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:16.371566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de3bbe-9283-46c1-9ae7-40391021c17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:16.390135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-66de3bbe-9283-46c1-9ae7-40391021c17a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:42:16.391345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-66de3bbe-9283-46c1-9ae7-40391021c17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:42:16.399533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:42:16.703874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb5f88-caab-49d8-b408-57890198a964 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:42:16.705551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eaeb5f88-caab-49d8-b408-57890198a964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:42:16.770281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:42:16.783458 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb5f88-caab-49d8-b408-57890198a964 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:42:16.787044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae06ed-c520-489b-8468-1d143ab7cc6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:42:16.787688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae06ed-c520-489b-8468-1d143ab7cc6d service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:16.787974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae06ed-c520-489b-8468-1d143ab7cc6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:16.788332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae06ed-c520-489b-8468-1d143ab7cc6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:16.797503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae06ed-c520-489b-8468-1d143ab7cc6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:42:16.798953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae06ed-c520-489b-8468-1d143ab7cc6d service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:42:16.809736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 772/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:42:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:42:18.878998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62cfe8-9f2c-495f-93a7-f5bfebf6a82f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:18.879674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62cfe8-9f2c-495f-93a7-f5bfebf6a82f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:18.879674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62cfe8-9f2c-495f-93a7-f5bfebf6a82f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:18.880937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:42:18.886691 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf15eca-14c5-4b13-b346-1650887cf522 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:18.887131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf15eca-14c5-4b13-b346-1650887cf522 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:18.887289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf15eca-14c5-4b13-b346-1650887cf522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:18.910379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-acf15eca-14c5-4b13-b346-1650887cf522 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:42:18.911610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-acf15eca-14c5-4b13-b346-1650887cf522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:42:18.920010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:42:19.228388 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62cfe8-9f2c-495f-93a7-f5bfebf6a82f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:42:19.230110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2d62cfe8-9f2c-495f-93a7-f5bfebf6a82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:42:19.237714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=84 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:42:19.244161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62cfe8-9f2c-495f-93a7-f5bfebf6a82f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:42:19.259148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf4c01-1c20-4eac-9991-15ff64ded1f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:42:19.259148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf4c01-1c20-4eac-9991-15ff64ded1f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:42:19.259148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf4c01-1c20-4eac-9991-15ff64ded1f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:42:19.259148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf4c01-1c20-4eac-9991-15ff64ded1f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:42:19.259592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf4c01-1c20-4eac-9991-15ff64ded1f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:42:19.260233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf4c01-1c20-4eac-9991-15ff64ded1f4 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:42:19.269791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 766/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:42:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:42:28.904511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=11 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:42:29.286861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:50.901137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd80e4e-a217-47ad-b046-2651205880d4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:50.902278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd80e4e-a217-47ad-b046-2651205880d4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:50.902782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd80e4e-a217-47ad-b046-2651205880d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:50.928290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-dcd80e4e-a217-47ad-b046-2651205880d4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:50.930181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dcd80e4e-a217-47ad-b046-2651205880d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:50.946400 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 774/1540] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:50.953658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2b77c-2f12-4d9e-91c9-5c253d241db9 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:50.954637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2b77c-2f12-4d9e-91c9-5c253d241db9 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:50.955146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2b77c-2f12-4d9e-91c9-5c253d241db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:50.978389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-99e2b77c-2f12-4d9e-91c9-5c253d241db9 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:50.981264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-99e2b77c-2f12-4d9e-91c9-5c253d241db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:50.991661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e4df-487b-4b84-85c7-c23c96852af9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:50.993601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e4df-487b-4b84-85c7-c23c96852af9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:50.994219 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e4df-487b-4b84-85c7-c23c96852af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.024852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 767/1541] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.026885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f1d2e4df-487b-4b84-85c7-c23c96852af9 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.028677 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f1d2e4df-487b-4b84-85c7-c23c96852af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.029115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbc4c6-dd56-42fc-b5f1-e3f4e308ad58 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.029425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbc4c6-dd56-42fc-b5f1-e3f4e308ad58 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.029698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbc4c6-dd56-42fc-b5f1-e3f4e308ad58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.045667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 775/1542] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.047316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9840b-788b-40b1-9bdb-03a23556374b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.048042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9840b-788b-40b1-9bdb-03a23556374b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.048556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9840b-788b-40b1-9bdb-03a23556374b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.062459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-6cbbc4c6-dd56-42fc-b5f1-e3f4e308ad58 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.064500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6cbbc4c6-dd56-42fc-b5f1-e3f4e308ad58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.086742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f3a9840b-788b-40b1-9bdb-03a23556374b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.088770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 768/1543] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:43:51.089286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f3a9840b-788b-40b1-9bdb-03a23556374b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.089670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbc4c6-dd56-42fc-b5f1-e3f4e308ad58 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.104309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbc4c6-dd56-42fc-b5f1-e3f4e308ad58 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.122009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d79247-6970-452f-a7b5-8f7e528ff217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.122707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d79247-6970-452f-a7b5-8f7e528ff217 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.122993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d79247-6970-452f-a7b5-8f7e528ff217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.123276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d79247-6970-452f-a7b5-8f7e528ff217 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.124270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d79247-6970-452f-a7b5-8f7e528ff217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:43:51.124571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d79247-6970-452f-a7b5-8f7e528ff217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.125666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: WARNING py.warnings [None req-18d79247-6970-452f-a7b5-8f7e528ff217 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:43:51.125666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: warnings.warn(msg) Jul 14 11:43:51.125666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: Jul 14 11:43:51.126560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d79247-6970-452f-a7b5-8f7e528ff217 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.128260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 776/1544] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.130269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4080066-e8a5-4748-b13c-12fe45a94a41 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.130824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4080066-e8a5-4748-b13c-12fe45a94a41 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.131332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4080066-e8a5-4748-b13c-12fe45a94a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.144242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 769/1545] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:43:51.145262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d79247-6970-452f-a7b5-8f7e528ff217 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.175555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d79247-6970-452f-a7b5-8f7e528ff217 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.181279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c4080066-e8a5-4748-b13c-12fe45a94a41 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.183327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c4080066-e8a5-4748-b13c-12fe45a94a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.187312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fca95-de2c-46a1-8c16-c1bf7a13ad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.187312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fca95-de2c-46a1-8c16-c1bf7a13ad2d admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.187312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fca95-de2c-46a1-8c16-c1bf7a13ad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.187312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fca95-de2c-46a1-8c16-c1bf7a13ad2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.187712 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fca95-de2c-46a1-8c16-c1bf7a13ad2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:43:51.187712 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fca95-de2c-46a1-8c16-c1bf7a13ad2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.188885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fca95-de2c-46a1-8c16-c1bf7a13ad2d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.211388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 777/1546] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.211927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 770/1547] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:43:51.213021 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4080066-e8a5-4748-b13c-12fe45a94a41 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.213740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adabf9-b873-412b-a529-e1643f3a4cd3 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.214032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adabf9-b873-412b-a529-e1643f3a4cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.214308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adabf9-b873-412b-a529-e1643f3a4cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.232633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4080066-e8a5-4748-b13c-12fe45a94a41 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.245921 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d97d73-785b-4a2a-9048-b05800068cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.246972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97d73-785b-4a2a-9048-b05800068cac admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.247479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97d73-785b-4a2a-9048-b05800068cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.247979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d97d73-785b-4a2a-9048-b05800068cac admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.248901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d97d73-785b-4a2a-9048-b05800068cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.250226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-d1adabf9-b873-412b-a529-e1643f3a4cd3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.250591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-e4d97d73-785b-4a2a-9048-b05800068cac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:43:51.250591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Jul 14 11:43:51.250591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: Jul 14 11:43:51.251460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d1adabf9-b873-412b-a529-e1643f3a4cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.252714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d97d73-785b-4a2a-9048-b05800068cac admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.264959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 771/1548] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.269182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1adabf9-b873-412b-a529-e1643f3a4cd3 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.279546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1adabf9-b873-412b-a529-e1643f3a4cd3 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.292426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e224aee-4601-45f9-bd9e-c0b01c712547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.293099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e224aee-4601-45f9-bd9e-c0b01c712547 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.293419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e224aee-4601-45f9-bd9e-c0b01c712547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.293695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e224aee-4601-45f9-bd9e-c0b01c712547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.294446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e224aee-4601-45f9-bd9e-c0b01c712547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:43:51.294752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e224aee-4601-45f9-bd9e-c0b01c712547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.296378 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e224aee-4601-45f9-bd9e-c0b01c712547 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.310041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 772/1549] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:43:51.310877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e224aee-4601-45f9-bd9e-c0b01c712547 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.321114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e224aee-4601-45f9-bd9e-c0b01c712547 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.327109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18462173-da8c-4685-9374-a40e45395fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.327923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18462173-da8c-4685-9374-a40e45395fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.328288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18462173-da8c-4685-9374-a40e45395fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.328641 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18462173-da8c-4685-9374-a40e45395fe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.329459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18462173-da8c-4685-9374-a40e45395fe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.332427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18462173-da8c-4685-9374-a40e45395fe4 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.433624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 778/1550] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/projects => generated 403 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:51.434981 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12169144-53fe-4efb-81bd-83852cf73fef None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.435416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12169144-53fe-4efb-81bd-83852cf73fef None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.435704 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12169144-53fe-4efb-81bd-83852cf73fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.456894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-12169144-53fe-4efb-81bd-83852cf73fef None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.458176 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-12169144-53fe-4efb-81bd-83852cf73fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.462515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 773/1551] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:51.464601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1d9b6-2142-4442-acd3-3082f92bd208 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.464899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1d9b6-2142-4442-acd3-3082f92bd208 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.465201 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1d9b6-2142-4442-acd3-3082f92bd208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.487935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c7b1d9b6-2142-4442-acd3-3082f92bd208 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.489058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c7b1d9b6-2142-4442-acd3-3082f92bd208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.524217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 779/1552] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.525543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28173f-5303-45f0-bfca-d5a20979b528 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.526320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28173f-5303-45f0-bfca-d5a20979b528 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.526597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28173f-5303-45f0-bfca-d5a20979b528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.545205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-bb28173f-5303-45f0-bfca-d5a20979b528 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.546415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bb28173f-5303-45f0-bfca-d5a20979b528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.563238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 774/1553] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.563238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1d9b6-2142-4442-acd3-3082f92bd208 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.577192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1d9b6-2142-4442-acd3-3082f92bd208 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.578089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7bc6a-2415-460c-8de8-2990761fd344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.578770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7bc6a-2415-460c-8de8-2990761fd344 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.579050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7bc6a-2415-460c-8de8-2990761fd344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.579321 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7bc6a-2415-460c-8de8-2990761fd344 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.580020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7bc6a-2415-460c-8de8-2990761fd344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.581969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7bc6a-2415-460c-8de8-2990761fd344 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.607018 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 780/1554] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.608315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90170c4a-8625-43f1-82e4-515db95edb5a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.608595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90170c4a-8625-43f1-82e4-515db95edb5a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.608865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90170c4a-8625-43f1-82e4-515db95edb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.626979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-90170c4a-8625-43f1-82e4-515db95edb5a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.628163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-90170c4a-8625-43f1-82e4-515db95edb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.636448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 781/1555] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.638719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc752c-9663-42a1-9b2e-f249a37150ed None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.638719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc752c-9663-42a1-9b2e-f249a37150ed None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.638719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc752c-9663-42a1-9b2e-f249a37150ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.656442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-10bc752c-9663-42a1-9b2e-f249a37150ed None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.657672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-10bc752c-9663-42a1-9b2e-f249a37150ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.665640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 782/1556] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.666961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc752c-9663-42a1-9b2e-f249a37150ed None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.677625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc752c-9663-42a1-9b2e-f249a37150ed None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.681668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 775/1557] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:51.684967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7bc6a-2415-460c-8de8-2990761fd344 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.685144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec79b70-1119-41ea-8987-4d24a6b6c012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.685557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec79b70-1119-41ea-8987-4d24a6b6c012 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.686019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec79b70-1119-41ea-8987-4d24a6b6c012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.686452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec79b70-1119-41ea-8987-4d24a6b6c012 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.687244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec79b70-1119-41ea-8987-4d24a6b6c012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.689597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec79b70-1119-41ea-8987-4d24a6b6c012 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.741632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:51.745476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7bc6a-2415-460c-8de8-2990761fd344 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.750550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4f008-8558-4f33-9065-0e3e8ec3eadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.755149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4f008-8558-4f33-9065-0e3e8ec3eadb admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.755149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4f008-8558-4f33-9065-0e3e8ec3eadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.755149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4f008-8558-4f33-9065-0e3e8ec3eadb admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.755492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 783/1558] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/users => generated 302 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:51.755607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4f008-8558-4f33-9065-0e3e8ec3eadb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.755717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec79b70-1119-41ea-8987-4d24a6b6c012 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.755817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: WARNING py.warnings [None req-15c4f008-8558-4f33-9065-0e3e8ec3eadb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:43:51.755817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: warnings.warn(msg) Jul 14 11:43:51.755817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: Jul 14 11:43:51.755817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4f008-8558-4f33-9065-0e3e8ec3eadb admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.764090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec79b70-1119-41ea-8987-4d24a6b6c012 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.768934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4dd7e-b76b-408f-a3f6-079054512f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.769637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4dd7e-b76b-408f-a3f6-079054512f83 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.769914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4dd7e-b76b-408f-a3f6-079054512f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.770185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4dd7e-b76b-408f-a3f6-079054512f83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.770977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4dd7e-b76b-408f-a3f6-079054512f83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:43:51.771272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4dd7e-b76b-408f-a3f6-079054512f83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.772876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4dd7e-b76b-408f-a3f6-079054512f83 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.783698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:51.788381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 784/1559] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:43:51.788700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c286ed-3578-4524-ac9f-7eba5aff9949 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.788983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c286ed-3578-4524-ac9f-7eba5aff9949 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.789589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c286ed-3578-4524-ac9f-7eba5aff9949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.814418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c1c286ed-3578-4524-ac9f-7eba5aff9949 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.815622 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c1c286ed-3578-4524-ac9f-7eba5aff9949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.824649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 785/1560] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.825960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6072429-0555-4024-addb-045abbbdd03b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.826306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6072429-0555-4024-addb-045abbbdd03b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.826583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6072429-0555-4024-addb-045abbbdd03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.846021 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d6072429-0555-4024-addb-045abbbdd03b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.847139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d6072429-0555-4024-addb-045abbbdd03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.855086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 786/1561] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.856316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36a00e-f26f-4fa0-970f-65d30e476fa0 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.856601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36a00e-f26f-4fa0-970f-65d30e476fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.856875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36a00e-f26f-4fa0-970f-65d30e476fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.873811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 776/1562] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:51.875959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690968db-9610-47e5-a0ec-b445a11a55da None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.876093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ef36a00e-f26f-4fa0-970f-65d30e476fa0 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.876194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690968db-9610-47e5-a0ec-b445a11a55da None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.876194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690968db-9610-47e5-a0ec-b445a11a55da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.876367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ef36a00e-f26f-4fa0-970f-65d30e476fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.883533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 787/1563] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.883533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36a00e-f26f-4fa0-970f-65d30e476fa0 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.894460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36a00e-f26f-4fa0-970f-65d30e476fa0 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.894710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-690968db-9610-47e5-a0ec-b445a11a55da None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:51.895941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-690968db-9610-47e5-a0ec-b445a11a55da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:51.900279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0a8d9-afd9-4f23-8f65-ded1c0b546ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.901536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0a8d9-afd9-4f23-8f65-ded1c0b546ab admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.902126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0a8d9-afd9-4f23-8f65-ded1c0b546ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.902126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0a8d9-afd9-4f23-8f65-ded1c0b546ab admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.902749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0a8d9-afd9-4f23-8f65-ded1c0b546ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.904658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0a8d9-afd9-4f23-8f65-ded1c0b546ab admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.904921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 777/1564] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:51.905825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690968db-9610-47e5-a0ec-b445a11a55da None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.915043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690968db-9610-47e5-a0ec-b445a11a55da None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.923772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4cd36-3647-4b2e-8acf-de65585d8149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.924462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4cd36-3647-4b2e-8acf-de65585d8149 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.924748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4cd36-3647-4b2e-8acf-de65585d8149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.925027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4cd36-3647-4b2e-8acf-de65585d8149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.925811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4cd36-3647-4b2e-8acf-de65585d8149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.927432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4cd36-3647-4b2e-8acf-de65585d8149 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:51.938380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 778/1565] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:51.939247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4cd36-3647-4b2e-8acf-de65585d8149 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:51.950317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4cd36-3647-4b2e-8acf-de65585d8149 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:51.957728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3101e7-8ade-4c99-91ee-f0767c60173e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:51.957728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3101e7-8ade-4c99-91ee-f0767c60173e admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:51.957728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3101e7-8ade-4c99-91ee-f0767c60173e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:51.957728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3101e7-8ade-4c99-91ee-f0767c60173e admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:51.958613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3101e7-8ade-4c99-91ee-f0767c60173e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:51.958711 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3101e7-8ade-4c99-91ee-f0767c60173e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.001370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 788/1566] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:52.001370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0a8d9-afd9-4f23-8f65-ded1c0b546ab admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.009065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0a8d9-afd9-4f23-8f65-ded1c0b546ab admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.014214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08831698-e411-4b9d-8b67-4fa37b2ea926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.014869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08831698-e411-4b9d-8b67-4fa37b2ea926 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.015159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08831698-e411-4b9d-8b67-4fa37b2ea926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.015434 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08831698-e411-4b9d-8b67-4fa37b2ea926 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.016156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08831698-e411-4b9d-8b67-4fa37b2ea926 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:43:52.016457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08831698-e411-4b9d-8b67-4fa37b2ea926 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.018186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08831698-e411-4b9d-8b67-4fa37b2ea926 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.031501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 789/1567] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:43:52.037432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08831698-e411-4b9d-8b67-4fa37b2ea926 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.042178 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08831698-e411-4b9d-8b67-4fa37b2ea926 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.046841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6d072-a1fd-41ad-a329-43ed16b60289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.047484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6d072-a1fd-41ad-a329-43ed16b60289 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.047779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6d072-a1fd-41ad-a329-43ed16b60289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.048160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6d072-a1fd-41ad-a329-43ed16b60289 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.048840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6d072-a1fd-41ad-a329-43ed16b60289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:43:52.049144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6d072-a1fd-41ad-a329-43ed16b60289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.050745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6d072-a1fd-41ad-a329-43ed16b60289 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.062370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 790/1568] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:43:52.063284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6d072-a1fd-41ad-a329-43ed16b60289 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.073851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6d072-a1fd-41ad-a329-43ed16b60289 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.078757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fb13e-2928-4d14-922c-b7fba609bb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.079414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fb13e-2928-4d14-922c-b7fba609bb21 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.079696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fb13e-2928-4d14-922c-b7fba609bb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.079971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fb13e-2928-4d14-922c-b7fba609bb21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.080692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fb13e-2928-4d14-922c-b7fba609bb21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.082336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fb13e-2928-4d14-922c-b7fba609bb21 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.085607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 779/1569] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:51 2022] POST /identity/v3/projects => generated 389 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:52.086584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3101e7-8ade-4c99-91ee-f0767c60173e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.095789 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 791/1570] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:52.095789 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fb13e-2928-4d14-922c-b7fba609bb21 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.106008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fb13e-2928-4d14-922c-b7fba609bb21 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.156333 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3101e7-8ade-4c99-91ee-f0767c60173e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.161100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7137b6f-5684-4e39-be1b-065f2048392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.161913 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7137b6f-5684-4e39-be1b-065f2048392a admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.162196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7137b6f-5684-4e39-be1b-065f2048392a admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.162588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7137b6f-5684-4e39-be1b-065f2048392a admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.163214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7137b6f-5684-4e39-be1b-065f2048392a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:43:52.163530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7137b6f-5684-4e39-be1b-065f2048392a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.165263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7137b6f-5684-4e39-be1b-065f2048392a admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.173223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b04930-c696-44a2-8d3b-feeeae401273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.173895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b04930-c696-44a2-8d3b-feeeae401273 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.174179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b04930-c696-44a2-8d3b-feeeae401273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.174459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b04930-c696-44a2-8d3b-feeeae401273 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.175184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b04930-c696-44a2-8d3b-feeeae401273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:43:52.175490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b04930-c696-44a2-8d3b-feeeae401273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.177075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b04930-c696-44a2-8d3b-feeeae401273 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.177298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 780/1571] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:43:52.178262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7137b6f-5684-4e39-be1b-065f2048392a admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.188811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7137b6f-5684-4e39-be1b-065f2048392a admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.193772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80565911-ad7c-44e3-b09c-56a4106c45df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.194457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80565911-ad7c-44e3-b09c-56a4106c45df admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.194736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80565911-ad7c-44e3-b09c-56a4106c45df admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.195010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80565911-ad7c-44e3-b09c-56a4106c45df admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.202769 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 792/1572] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:43:52.202937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80565911-ad7c-44e3-b09c-56a4106c45df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.203101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b04930-c696-44a2-8d3b-feeeae401273 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.203198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80565911-ad7c-44e3-b09c-56a4106c45df admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.209555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b04930-c696-44a2-8d3b-feeeae401273 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.213310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 781/1573] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:52.213310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80565911-ad7c-44e3-b09c-56a4106c45df admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.217400 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d24cd6-904d-4452-bd78-6c0a9af3ea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.218141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d24cd6-904d-4452-bd78-6c0a9af3ea51 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.218420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d24cd6-904d-4452-bd78-6c0a9af3ea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.218693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d24cd6-904d-4452-bd78-6c0a9af3ea51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.219453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d24cd6-904d-4452-bd78-6c0a9af3ea51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.220878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80565911-ad7c-44e3-b09c-56a4106c45df admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.221043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d24cd6-904d-4452-bd78-6c0a9af3ea51 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.225586 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2844d-db16-46ac-b103-dce7c67dd892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.226238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2844d-db16-46ac-b103-dce7c67dd892 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.226886 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2844d-db16-46ac-b103-dce7c67dd892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.226886 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2844d-db16-46ac-b103-dce7c67dd892 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.227458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2844d-db16-46ac-b103-dce7c67dd892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.229380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2844d-db16-46ac-b103-dce7c67dd892 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.232383 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 793/1574] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:52.234365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d24cd6-904d-4452-bd78-6c0a9af3ea51 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.245417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d24cd6-904d-4452-bd78-6c0a9af3ea51 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.250180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47a0d1-88c2-4991-b3ba-717f35bce986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.250832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47a0d1-88c2-4991-b3ba-717f35bce986 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.251136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47a0d1-88c2-4991-b3ba-717f35bce986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.251432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47a0d1-88c2-4991-b3ba-717f35bce986 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.252088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47a0d1-88c2-4991-b3ba-717f35bce986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.254109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47a0d1-88c2-4991-b3ba-717f35bce986 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.331446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 782/1575] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:52 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:52.332362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2844d-db16-46ac-b103-dce7c67dd892 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.342200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 794/1576] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:52 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:52.342912 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47a0d1-88c2-4991-b3ba-717f35bce986 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.398968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:52.401893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2844d-db16-46ac-b103-dce7c67dd892 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.412730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47a0d1-88c2-4991-b3ba-717f35bce986 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.413514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:52.417425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fe345-6505-4021-b68b-0c3e23ac752b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.423302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe345-6505-4021-b68b-0c3e23ac752b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.423302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe345-6505-4021-b68b-0c3e23ac752b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.423302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe345-6505-4021-b68b-0c3e23ac752b admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.423302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fe345-6505-4021-b68b-0c3e23ac752b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.423302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fe345-6505-4021-b68b-0c3e23ac752b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.430845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 795/1577] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:52.431715 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fe345-6505-4021-b68b-0c3e23ac752b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.441788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fe345-6505-4021-b68b-0c3e23ac752b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.447415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9b69a-41ab-4a5d-87c7-09575a2729f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.452075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b69a-41ab-4a5d-87c7-09575a2729f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.452075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b69a-41ab-4a5d-87c7-09575a2729f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.452075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b69a-41ab-4a5d-87c7-09575a2729f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.452075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9b69a-41ab-4a5d-87c7-09575a2729f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.452075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9b69a-41ab-4a5d-87c7-09575a2729f0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.460716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630df1ff-7b63-4261-89a6-e71a6981c718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.461424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630df1ff-7b63-4261-89a6-e71a6981c718 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.461709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630df1ff-7b63-4261-89a6-e71a6981c718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.461981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630df1ff-7b63-4261-89a6-e71a6981c718 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.462681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630df1ff-7b63-4261-89a6-e71a6981c718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.464706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630df1ff-7b63-4261-89a6-e71a6981c718 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.661393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 796/1578] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:52 2022] POST /identity/v3/projects => generated 371 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:52.662207 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9b69a-41ab-4a5d-87c7-09575a2729f0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.692252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 783/1579] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:52 2022] POST /identity/v3/users => generated 295 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:52.692252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630df1ff-7b63-4261-89a6-e71a6981c718 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.699766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630df1ff-7b63-4261-89a6-e71a6981c718 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.726143 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9b69a-41ab-4a5d-87c7-09575a2729f0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.731414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9abaa-402f-4d87-a7c8-684c5eb56ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.732157 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9abaa-402f-4d87-a7c8-684c5eb56ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.732444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9abaa-402f-4d87-a7c8-684c5eb56ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.732721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9abaa-402f-4d87-a7c8-684c5eb56ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.733447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9abaa-402f-4d87-a7c8-684c5eb56ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.735356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9abaa-402f-4d87-a7c8-684c5eb56ee7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.757082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577edc1-6105-4a5e-a2e1-7dc8a470271a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.759153 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577edc1-6105-4a5e-a2e1-7dc8a470271a admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.759153 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577edc1-6105-4a5e-a2e1-7dc8a470271a admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.759153 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577edc1-6105-4a5e-a2e1-7dc8a470271a admin admin] PATH_INFO: `/v3/projects/92d73ccf738243868717597ce7e03627/users/78e644cc7e6d46aba520d3454607416b/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.773047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577edc1-6105-4a5e-a2e1-7dc8a470271a admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d73ccf738243868717597ce7e03627, user_id=78e644cc7e6d46aba520d3454607416b, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.776044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577edc1-6105-4a5e-a2e1-7dc8a470271a admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.793725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 797/1580] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:52 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:52.794614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9abaa-402f-4d87-a7c8-684c5eb56ee7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.844007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 784/1581] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:52 2022] PUT /identity/v3/projects/92d73ccf738243868717597ce7e03627/users/78e644cc7e6d46aba520d3454607416b/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:52.846896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577edc1-6105-4a5e-a2e1-7dc8a470271a admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.858289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9abaa-402f-4d87-a7c8-684c5eb56ee7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.906456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:52.919855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2377f2-df5f-40e6-926e-66c236abe8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.919855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2377f2-df5f-40e6-926e-66c236abe8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.919855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2377f2-df5f-40e6-926e-66c236abe8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.919855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2377f2-df5f-40e6-926e-66c236abe8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.920314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2377f2-df5f-40e6-926e-66c236abe8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.920314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2377f2-df5f-40e6-926e-66c236abe8a4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.921958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577edc1-6105-4a5e-a2e1-7dc8a470271a admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.926701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282230e5-a364-4d10-9c41-095d1c33671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.927544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282230e5-a364-4d10-9c41-095d1c33671d admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.927830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282230e5-a364-4d10-9c41-095d1c33671d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.928105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282230e5-a364-4d10-9c41-095d1c33671d admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.928800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282230e5-a364-4d10-9c41-095d1c33671d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.932494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 798/1582] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:52.932494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2377f2-df5f-40e6-926e-66c236abe8a4 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:52.932749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282230e5-a364-4d10-9c41-095d1c33671d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:52.943098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2377f2-df5f-40e6-926e-66c236abe8a4 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:52.947783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36bbe3-c7bc-41d9-8c09-7325b0d35e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:52.948442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36bbe3-c7bc-41d9-8c09-7325b0d35e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:52.948732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36bbe3-c7bc-41d9-8c09-7325b0d35e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:52.949031 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36bbe3-c7bc-41d9-8c09-7325b0d35e00 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:52.949762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36bbe3-c7bc-41d9-8c09-7325b0d35e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:52.951660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36bbe3-c7bc-41d9-8c09-7325b0d35e00 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.039933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 799/1583] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:52 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:53.040243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 785/1584] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:52 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:53.040695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36bbe3-c7bc-41d9-8c09-7325b0d35e00 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.040800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282230e5-a364-4d10-9c41-095d1c33671d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.049264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282230e5-a364-4d10-9c41-095d1c33671d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.050998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36bbe3-c7bc-41d9-8c09-7325b0d35e00 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.053889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2a126-a93c-44df-ae12-b354d1d498cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.063042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2a126-a93c-44df-ae12-b354d1d498cf admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.063042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2a126-a93c-44df-ae12-b354d1d498cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.063042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2a126-a93c-44df-ae12-b354d1d498cf admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.063042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2a126-a93c-44df-ae12-b354d1d498cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.063456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116e2ab-c2e8-400c-af7f-8ebe8804ebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.063456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116e2ab-c2e8-400c-af7f-8ebe8804ebe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.063456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116e2ab-c2e8-400c-af7f-8ebe8804ebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.063456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116e2ab-c2e8-400c-af7f-8ebe8804ebe9 admin admin] PATH_INFO: `/v3/projects/48a88ac7db8a481ca03508d315a0bc56/users/246f07024efa444da74f2b898f426b46/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.063803 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2a126-a93c-44df-ae12-b354d1d498cf admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.072927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116e2ab-c2e8-400c-af7f-8ebe8804ebe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a88ac7db8a481ca03508d315a0bc56, user_id=246f07024efa444da74f2b898f426b46, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.074007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-4116e2ab-c2e8-400c-af7f-8ebe8804ebe9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:43:53.074007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Jul 14 11:43:53.074007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: Jul 14 11:43:53.075958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116e2ab-c2e8-400c-af7f-8ebe8804ebe9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.140909 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:53.144979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 800/1585] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:53 2022] PUT /identity/v3/projects/48a88ac7db8a481ca03508d315a0bc56/users/246f07024efa444da74f2b898f426b46/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:53.145871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116e2ab-c2e8-400c-af7f-8ebe8804ebe9 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.149671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 786/1586] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:53 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:53.149671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2a126-a93c-44df-ae12-b354d1d498cf admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.157645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2a126-a93c-44df-ae12-b354d1d498cf admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.210985 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116e2ab-c2e8-400c-af7f-8ebe8804ebe9 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.213911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea55dfd-3b35-4bb1-b4ec-bc7b04f85db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.214841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea55dfd-3b35-4bb1-b4ec-bc7b04f85db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.215488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea55dfd-3b35-4bb1-b4ec-bc7b04f85db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.215904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea55dfd-3b35-4bb1-b4ec-bc7b04f85db7 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.216628 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea55dfd-3b35-4bb1-b4ec-bc7b04f85db7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.216825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5adb2-e816-4121-8567-9431ad9d09bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.217585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5adb2-e816-4121-8567-9431ad9d09bf admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.217817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5adb2-e816-4121-8567-9431ad9d09bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.218094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5adb2-e816-4121-8567-9431ad9d09bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.218591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea55dfd-3b35-4bb1-b4ec-bc7b04f85db7 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.218826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5adb2-e816-4121-8567-9431ad9d09bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.220409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5adb2-e816-4121-8567-9431ad9d09bf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.231743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 801/1587] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:53.233040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8fae2-36d5-4ff6-bad3-1898e2e67d56 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.233355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8fae2-36d5-4ff6-bad3-1898e2e67d56 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.233635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8fae2-36d5-4ff6-bad3-1898e2e67d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.261460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-abe8fae2-36d5-4ff6-bad3-1898e2e67d56 None None] MFA Rules not processed for user `78e644cc7e6d46aba520d3454607416b`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:53.262680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-abe8fae2-36d5-4ff6-bad3-1898e2e67d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:53.328347 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 802/1588] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:53.328748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 787/1589] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:53 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:53.329204 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8fae2-36d5-4ff6-bad3-1898e2e67d56 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.329727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea55dfd-3b35-4bb1-b4ec-bc7b04f85db7 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.341281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea55dfd-3b35-4bb1-b4ec-bc7b04f85db7 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.341844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8fae2-36d5-4ff6-bad3-1898e2e67d56 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.346271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8bdb-6303-4d3b-9c5f-cab945397f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.346622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e02202-cc6d-4079-a86d-ceb5f09074c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.347149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8bdb-6303-4d3b-9c5f-cab945397f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.347271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e02202-cc6d-4079-a86d-ceb5f09074c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.347426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8bdb-6303-4d3b-9c5f-cab945397f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.347560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e02202-cc6d-4079-a86d-ceb5f09074c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.347697 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8bdb-6303-4d3b-9c5f-cab945397f2b admin admin] PATH_INFO: `/v3/projects/00a4eb6dbc8b47d39835a10a93e586e9/users/846c97e0d4d84f738df73256856b2498/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.347840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e02202-cc6d-4079-a86d-ceb5f09074c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.348621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e02202-cc6d-4079-a86d-ceb5f09074c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.350366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e02202-cc6d-4079-a86d-ceb5f09074c0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.360934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 788/1590] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:53.361654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e02202-cc6d-4079-a86d-ceb5f09074c0 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.366145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f8bdb-6303-4d3b-9c5f-cab945397f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a4eb6dbc8b47d39835a10a93e586e9, user_id=846c97e0d4d84f738df73256856b2498, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.369589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f8bdb-6303-4d3b-9c5f-cab945397f2b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.372513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e02202-cc6d-4079-a86d-ceb5f09074c0 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.378030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250ee5f-83cb-441e-8aa3-ca1a0adcb864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.378729 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250ee5f-83cb-441e-8aa3-ca1a0adcb864 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.379013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250ee5f-83cb-441e-8aa3-ca1a0adcb864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.379287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250ee5f-83cb-441e-8aa3-ca1a0adcb864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.380013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250ee5f-83cb-441e-8aa3-ca1a0adcb864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.381676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250ee5f-83cb-441e-8aa3-ca1a0adcb864 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.393471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 789/1591] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:53.394767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5cd79-bb3f-42e5-9ab4-1f9ce7073b80 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.395051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5cd79-bb3f-42e5-9ab4-1f9ce7073b80 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.395324 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5cd79-bb3f-42e5-9ab4-1f9ce7073b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.412431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-40c5cd79-bb3f-42e5-9ab4-1f9ce7073b80 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:53.413659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-40c5cd79-bb3f-42e5-9ab4-1f9ce7073b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:53.421923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 790/1592] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:53.430666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5cd79-bb3f-42e5-9ab4-1f9ce7073b80 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.432514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5cd79-bb3f-42e5-9ab4-1f9ce7073b80 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.437130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc2a8f-6adc-4d22-b2dc-8de987604446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.437812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc2a8f-6adc-4d22-b2dc-8de987604446 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.438093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc2a8f-6adc-4d22-b2dc-8de987604446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.438367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc2a8f-6adc-4d22-b2dc-8de987604446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.439090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc2a8f-6adc-4d22-b2dc-8de987604446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.440685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc2a8f-6adc-4d22-b2dc-8de987604446 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.453974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 791/1593] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:53.453974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc2a8f-6adc-4d22-b2dc-8de987604446 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.454396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 803/1594] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:53 2022] PUT /identity/v3/projects/00a4eb6dbc8b47d39835a10a93e586e9/users/846c97e0d4d84f738df73256856b2498/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:53.454396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8bdb-6303-4d3b-9c5f-cab945397f2b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.466549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc2a8f-6adc-4d22-b2dc-8de987604446 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.475083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:53.521582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb82fb-b137-43b0-be13-704803fa1532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.522258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb82fb-b137-43b0-be13-704803fa1532 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.522542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb82fb-b137-43b0-be13-704803fa1532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.522819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb82fb-b137-43b0-be13-704803fa1532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.523852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb82fb-b137-43b0-be13-704803fa1532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.526019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb82fb-b137-43b0-be13-704803fa1532 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.527104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8bdb-6303-4d3b-9c5f-cab945397f2b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.532551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfb6d5-8af8-4456-8968-0f87e50cfadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.533250 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb6d5-8af8-4456-8968-0f87e50cfadc admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.533538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb6d5-8af8-4456-8968-0f87e50cfadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.533812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb6d5-8af8-4456-8968-0f87e50cfadc admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.534514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfb6d5-8af8-4456-8968-0f87e50cfadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.536505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfb6d5-8af8-4456-8968-0f87e50cfadc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.537552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 792/1595] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:53.538535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb82fb-b137-43b0-be13-704803fa1532 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.548823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb82fb-b137-43b0-be13-704803fa1532 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.554720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a7b19-08f4-4d63-9048-c5b4823068d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.555384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a7b19-08f4-4d63-9048-c5b4823068d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.555670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a7b19-08f4-4d63-9048-c5b4823068d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.556064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a7b19-08f4-4d63-9048-c5b4823068d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.556691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a7b19-08f4-4d63-9048-c5b4823068d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.558343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a7b19-08f4-4d63-9048-c5b4823068d1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.568432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 793/1596] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:53.569401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a7b19-08f4-4d63-9048-c5b4823068d1 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.579576 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a7b19-08f4-4d63-9048-c5b4823068d1 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.584286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f372c5f-20ab-4564-bf7b-1e1330b292f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.584924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f372c5f-20ab-4564-bf7b-1e1330b292f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.585240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f372c5f-20ab-4564-bf7b-1e1330b292f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.585518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f372c5f-20ab-4564-bf7b-1e1330b292f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.586231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f372c5f-20ab-4564-bf7b-1e1330b292f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.587790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f372c5f-20ab-4564-bf7b-1e1330b292f8 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.598760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 794/1597] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:53.599700 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f372c5f-20ab-4564-bf7b-1e1330b292f8 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.609236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f372c5f-20ab-4564-bf7b-1e1330b292f8 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.613993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadade7a-0522-401d-9804-9d08a6ed705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.614616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadade7a-0522-401d-9804-9d08a6ed705c admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.614900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadade7a-0522-401d-9804-9d08a6ed705c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.615176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadade7a-0522-401d-9804-9d08a6ed705c admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.615882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadade7a-0522-401d-9804-9d08a6ed705c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.617505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadade7a-0522-401d-9804-9d08a6ed705c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.627525 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 795/1598] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:53.628316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadade7a-0522-401d-9804-9d08a6ed705c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.641559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:53.653656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 804/1599] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:53 2022] POST /identity/v3/projects => generated 405 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:53.654902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfb6d5-8af8-4456-8968-0f87e50cfadc admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.686356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadade7a-0522-401d-9804-9d08a6ed705c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.719563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfb6d5-8af8-4456-8968-0f87e50cfadc admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.724402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecd405-5542-41ba-98c9-f5534441865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.725083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd405-5542-41ba-98c9-f5534441865e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.725393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd405-5542-41ba-98c9-f5534441865e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.725666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd405-5542-41ba-98c9-f5534441865e admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.726398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecd405-5542-41ba-98c9-f5534441865e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.728014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecd405-5542-41ba-98c9-f5534441865e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.735582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9c30b-b722-4d36-b603-51a92e20985e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.736222 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9c30b-b722-4d36-b603-51a92e20985e service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.736504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9c30b-b722-4d36-b603-51a92e20985e service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.736774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9c30b-b722-4d36-b603-51a92e20985e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.738210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 805/1600] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:53.739582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dc244-d133-42f3-813b-16830d88e0f4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.739865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dc244-d133-42f3-813b-16830d88e0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.740165 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dc244-d133-42f3-813b-16830d88e0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.746633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9c30b-b722-4d36-b603-51a92e20985e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.748067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9c30b-b722-4d36-b603-51a92e20985e service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.758032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 796/1601] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:43:53.758793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9c30b-b722-4d36-b603-51a92e20985e service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.767474 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-418dc244-d133-42f3-813b-16830d88e0f4 None None] MFA Rules not processed for user `846c97e0d4d84f738df73256856b2498`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:53.768689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-418dc244-d133-42f3-813b-16830d88e0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:53.769062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9c30b-b722-4d36-b603-51a92e20985e service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.776091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394474-65ab-4385-90a1-5b4226f26f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.777189 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394474-65ab-4385-90a1-5b4226f26f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.777352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394474-65ab-4385-90a1-5b4226f26f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.777587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394474-65ab-4385-90a1-5b4226f26f91 admin admin] PATH_INFO: `/v3/projects/81777430895b432999f69033667e59f1/users/899cbac79d874b308b500651bb40b2a7/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.792941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394474-65ab-4385-90a1-5b4226f26f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81777430895b432999f69033667e59f1, user_id=899cbac79d874b308b500651bb40b2a7, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.795613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394474-65ab-4385-90a1-5b4226f26f91 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.841892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 806/1602] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:53.842878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418dc244-d133-42f3-813b-16830d88e0f4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.855620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418dc244-d133-42f3-813b-16830d88e0f4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.860298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ecc08-0fb2-4c57-92c7-19953166412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.860964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ecc08-0fb2-4c57-92c7-19953166412a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.861278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ecc08-0fb2-4c57-92c7-19953166412a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.861577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ecc08-0fb2-4c57-92c7-19953166412a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.865971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ecc08-0fb2-4c57-92c7-19953166412a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.865971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ecc08-0fb2-4c57-92c7-19953166412a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.878623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 797/1603] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:53 2022] PUT /identity/v3/projects/81777430895b432999f69033667e59f1/users/899cbac79d874b308b500651bb40b2a7/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:53.878623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394474-65ab-4385-90a1-5b4226f26f91 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.883234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 807/1604] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:53.884065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ecc08-0fb2-4c57-92c7-19953166412a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:53.895525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ecc08-0fb2-4c57-92c7-19953166412a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.900847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:53.959780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394474-65ab-4385-90a1-5b4226f26f91 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:53.965927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d849455-6b94-4a18-9704-045ca32a1edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.966723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d3381-7e03-48ce-afd0-5b91c64a4caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:53.966901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d849455-6b94-4a18-9704-045ca32a1edc admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.967380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d849455-6b94-4a18-9704-045ca32a1edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.967703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d849455-6b94-4a18-9704-045ca32a1edc admin admin] PATH_INFO: `/v3/projects/5bb446eb04c04b73aacbcf5e605a6c76/users/760c5416ab5c44a59eb03971c2252725/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.967818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d3381-7e03-48ce-afd0-5b91c64a4caf admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:53.967987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d3381-7e03-48ce-afd0-5b91c64a4caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:53.968286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d3381-7e03-48ce-afd0-5b91c64a4caf admin admin] PATH_INFO: `/v3/projects/b2c5dbb0443e40109e2e82c19464128c/users/28b2a44e61794d32bf3b53633657077b/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:53.991583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d849455-6b94-4a18-9704-045ca32a1edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb446eb04c04b73aacbcf5e605a6c76, user_id=760c5416ab5c44a59eb03971c2252725, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.991583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d849455-6b94-4a18-9704-045ca32a1edc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:53.992160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d3381-7e03-48ce-afd0-5b91c64a4caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c5dbb0443e40109e2e82c19464128c, user_id=28b2a44e61794d32bf3b53633657077b, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:53.992325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d3381-7e03-48ce-afd0-5b91c64a4caf admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:54.117506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 808/1605] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:53 2022] PUT /identity/v3/projects/5bb446eb04c04b73aacbcf5e605a6c76/users/760c5416ab5c44a59eb03971c2252725/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:54.117710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 798/1606] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:53 2022] PUT /identity/v3/projects/b2c5dbb0443e40109e2e82c19464128c/users/28b2a44e61794d32bf3b53633657077b/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:54.118161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d849455-6b94-4a18-9704-045ca32a1edc admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:54.118264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d3381-7e03-48ce-afd0-5b91c64a4caf admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:54.151647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:54.181629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d3381-7e03-48ce-afd0-5b91c64a4caf admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:54.187044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997032c-9c14-44e0-877d-53599b79e2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:54.187940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997032c-9c14-44e0-877d-53599b79e2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.188221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997032c-9c14-44e0-877d-53599b79e2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.188498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997032c-9c14-44e0-877d-53599b79e2fd admin admin] PATH_INFO: `/v3/projects/5eb85e33b57d4660846c2dcd31bce190/users/39b601af0566421c9450ea89cc708d16/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.191049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d849455-6b94-4a18-9704-045ca32a1edc admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:54.195645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4af18-5d1f-40df-978c-48b5ba31a3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:54.196314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4af18-5d1f-40df-978c-48b5ba31a3bd admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.196666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4af18-5d1f-40df-978c-48b5ba31a3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.196869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4af18-5d1f-40df-978c-48b5ba31a3bd admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.197622 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4af18-5d1f-40df-978c-48b5ba31a3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:54.199585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4af18-5d1f-40df-978c-48b5ba31a3bd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:54.203439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997032c-9c14-44e0-877d-53599b79e2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb85e33b57d4660846c2dcd31bce190, user_id=39b601af0566421c9450ea89cc708d16, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:54.206818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997032c-9c14-44e0-877d-53599b79e2fd admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:54.290166 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 799/1607] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:54 2022] PUT /identity/v3/projects/5eb85e33b57d4660846c2dcd31bce190/users/39b601af0566421c9450ea89cc708d16/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:54.293075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997032c-9c14-44e0-877d-53599b79e2fd admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:54.301927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 809/1608] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:54 2022] POST /identity/v3/users => generated 309 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:54.303597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84f3ff-d13b-4710-a0c1-9ad5f244711f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.304091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84f3ff-d13b-4710-a0c1-9ad5f244711f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.304805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84f3ff-d13b-4710-a0c1-9ad5f244711f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.326380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2a84f3ff-d13b-4710-a0c1-9ad5f244711f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:54.327631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2a84f3ff-d13b-4710-a0c1-9ad5f244711f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:54.384118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997032c-9c14-44e0-877d-53599b79e2fd admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:54.393406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1e1a0-f59c-4927-b592-0615aa845d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:54.394098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1e1a0-f59c-4927-b592-0615aa845d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.394389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1e1a0-f59c-4927-b592-0615aa845d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.394827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1e1a0-f59c-4927-b592-0615aa845d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.395543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1e1a0-f59c-4927-b592-0615aa845d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:54.397556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1e1a0-f59c-4927-b592-0615aa845d0c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:54.411712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 810/1609] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:54.413056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84f3ff-d13b-4710-a0c1-9ad5f244711f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:54.425826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84f3ff-d13b-4710-a0c1-9ad5f244711f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:54.431097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a39ff2-a86b-4d0c-83b4-8f753943e81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:54.432788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39ff2-a86b-4d0c-83b4-8f753943e81f admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.433018 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39ff2-a86b-4d0c-83b4-8f753943e81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.433261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39ff2-a86b-4d0c-83b4-8f753943e81f admin admin] PATH_INFO: `/v3/projects/f274d555f5c746cc885394c699b3d767/users/8773c88f88614a9d903f667305a217e0/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.449048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a39ff2-a86b-4d0c-83b4-8f753943e81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f274d555f5c746cc885394c699b3d767, user_id=8773c88f88614a9d903f667305a217e0, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:54.452703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a39ff2-a86b-4d0c-83b4-8f753943e81f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:54.497435 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 800/1610] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:54 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:54.498819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcb120-7065-468e-bfb0-e1aaa5470f78 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.501496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcb120-7065-468e-bfb0-e1aaa5470f78 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.501496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcb120-7065-468e-bfb0-e1aaa5470f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.517644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-4bfcb120-7065-468e-bfb0-e1aaa5470f78 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:54.518750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4bfcb120-7065-468e-bfb0-e1aaa5470f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:54.547173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:54.555287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 811/1611] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:54 2022] PUT /identity/v3/projects/f274d555f5c746cc885394c699b3d767/users/8773c88f88614a9d903f667305a217e0/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:54.555287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe54a77-18bf-4cbd-a63c-558b81452d18 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.555287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe54a77-18bf-4cbd-a63c-558b81452d18 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.555671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe54a77-18bf-4cbd-a63c-558b81452d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.585069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2fe54a77-18bf-4cbd-a63c-558b81452d18 None None] MFA Rules not processed for user `28b2a44e61794d32bf3b53633657077b`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:54.587598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2fe54a77-18bf-4cbd-a63c-558b81452d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:54.593504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 801/1612] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:54.593504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aba25-6273-4860-9898-f6dc1977449c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.593504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aba25-6273-4860-9898-f6dc1977449c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.593504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aba25-6273-4860-9898-f6dc1977449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.628197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-885aba25-6273-4860-9898-f6dc1977449c None None] MFA Rules not processed for user `39b601af0566421c9450ea89cc708d16`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:54.628602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-885aba25-6273-4860-9898-f6dc1977449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:54.680826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 812/1613] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:54 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:54.685508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe54a77-18bf-4cbd-a63c-558b81452d18 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:54.686149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:54.722104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 802/1614] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:54.723032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aba25-6273-4860-9898-f6dc1977449c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:54.746979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe54a77-18bf-4cbd-a63c-558b81452d18 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:54.753923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c16233-0e37-4863-b934-2a9628f51d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:54.754611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c16233-0e37-4863-b934-2a9628f51d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.754897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c16233-0e37-4863-b934-2a9628f51d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.755172 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c16233-0e37-4863-b934-2a9628f51d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.755920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c16233-0e37-4863-b934-2a9628f51d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:54.757783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c16233-0e37-4863-b934-2a9628f51d59 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:54.772933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 813/1615] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:54.772933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c16233-0e37-4863-b934-2a9628f51d59 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:54.783694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c16233-0e37-4863-b934-2a9628f51d59 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:54.789044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe09e1-0e9d-4f0e-a51e-019c1ff5047e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:54.789768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe09e1-0e9d-4f0e-a51e-019c1ff5047e admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.790071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe09e1-0e9d-4f0e-a51e-019c1ff5047e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.790341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe09e1-0e9d-4f0e-a51e-019c1ff5047e admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.791096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe09e1-0e9d-4f0e-a51e-019c1ff5047e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:54.791806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aba25-6273-4860-9898-f6dc1977449c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:54.793601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe09e1-0e9d-4f0e-a51e-019c1ff5047e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:54.796847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede35d14-d38f-49b5-bdf9-7dce44881785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:54.797935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede35d14-d38f-49b5-bdf9-7dce44881785 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.798361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede35d14-d38f-49b5-bdf9-7dce44881785 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.801945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede35d14-d38f-49b5-bdf9-7dce44881785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.812211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede35d14-d38f-49b5-bdf9-7dce44881785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:54.814148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede35d14-d38f-49b5-bdf9-7dce44881785 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:54.827117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 803/1616] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:43:54.830296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69451cf-7efc-445a-b00f-bb6d8b503a02 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.830296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69451cf-7efc-445a-b00f-bb6d8b503a02 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.830296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69451cf-7efc-445a-b00f-bb6d8b503a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.848735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e69451cf-7efc-445a-b00f-bb6d8b503a02 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:54.850155 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e69451cf-7efc-445a-b00f-bb6d8b503a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:54.862797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 804/1617] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:54.862797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69451cf-7efc-445a-b00f-bb6d8b503a02 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:54.874569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69451cf-7efc-445a-b00f-bb6d8b503a02 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:54.878929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5dfd2-3a7a-45f5-9ddb-0b8e58becc5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:54.879595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5dfd2-3a7a-45f5-9ddb-0b8e58becc5d service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.879876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5dfd2-3a7a-45f5-9ddb-0b8e58becc5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.880160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5dfd2-3a7a-45f5-9ddb-0b8e58becc5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.891205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5dfd2-3a7a-45f5-9ddb-0b8e58becc5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:54.892655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5dfd2-3a7a-45f5-9ddb-0b8e58becc5d service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:54.902558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 814/1618] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:54 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:54.902558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe09e1-0e9d-4f0e-a51e-019c1ff5047e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:54.908247 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe09e1-0e9d-4f0e-a51e-019c1ff5047e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:54.909484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 805/1619] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:43:54.910288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5dfd2-3a7a-45f5-9ddb-0b8e58becc5d service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:54.921357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5dfd2-3a7a-45f5-9ddb-0b8e58becc5d service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:54.928739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a245e-38aa-40ba-875c-de8e61cdbc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:54.930243 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a245e-38aa-40ba-875c-de8e61cdbc92 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.939665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a245e-38aa-40ba-875c-de8e61cdbc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.939665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a245e-38aa-40ba-875c-de8e61cdbc92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.939665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a245e-38aa-40ba-875c-de8e61cdbc92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:54.940000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d60cc9-f39e-482c-b2b3-403c4a6cfa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:54.940000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60cc9-f39e-482c-b2b3-403c4a6cfa39 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:54.940000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60cc9-f39e-482c-b2b3-403c4a6cfa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:54.940000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60cc9-f39e-482c-b2b3-403c4a6cfa39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:54.940503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a245e-38aa-40ba-875c-de8e61cdbc92 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:54.940602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d60cc9-f39e-482c-b2b3-403c4a6cfa39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:54.940602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d60cc9-f39e-482c-b2b3-403c4a6cfa39 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.091161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 806/1620] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:54 2022] POST /identity/v3/projects => generated 371 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:55.092179 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d60cc9-f39e-482c-b2b3-403c4a6cfa39 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.093447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 815/1621] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:43:54 2022] POST /identity/v3/projects => generated 365 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:55.094259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a245e-38aa-40ba-875c-de8e61cdbc92 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.183505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:55.189351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d60cc9-f39e-482c-b2b3-403c4a6cfa39 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:55.200828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a245e-38aa-40ba-875c-de8e61cdbc92 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:55.206255 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903544a-694b-4912-94fc-67cb864785c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:55.208082 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903544a-694b-4912-94fc-67cb864785c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:55.208691 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903544a-694b-4912-94fc-67cb864785c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:55.209316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903544a-694b-4912-94fc-67cb864785c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:55.238888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:55.294923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d6ea4-d2bc-433a-95ac-ef3c4a83fdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:55.294923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d6ea4-d2bc-433a-95ac-ef3c4a83fdb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:55.294923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d6ea4-d2bc-433a-95ac-ef3c4a83fdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:55.294923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d6ea4-d2bc-433a-95ac-ef3c4a83fdb0 admin admin] PATH_INFO: `/v3/projects/d235718f97954ccb86c48fd996269c96/users/b5582f630d0648e7b159ad7142bc7c70/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:55.309615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d6ea4-d2bc-433a-95ac-ef3c4a83fdb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d235718f97954ccb86c48fd996269c96, user_id=b5582f630d0648e7b159ad7142bc7c70, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:55.314084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d6ea4-d2bc-433a-95ac-ef3c4a83fdb0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.339464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903544a-694b-4912-94fc-67cb864785c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:55.341201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903544a-694b-4912-94fc-67cb864785c1 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.356484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 816/1622] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:43:55.358059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903544a-694b-4912-94fc-67cb864785c1 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.371668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903544a-694b-4912-94fc-67cb864785c1 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:55.380699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810af98c-4f1a-4362-b725-915f682c0993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:55.381577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810af98c-4f1a-4362-b725-915f682c0993 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:55.381912 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810af98c-4f1a-4362-b725-915f682c0993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:55.382383 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810af98c-4f1a-4362-b725-915f682c0993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:55.386878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810af98c-4f1a-4362-b725-915f682c0993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:55.387150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810af98c-4f1a-4362-b725-915f682c0993 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.408784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 817/1623] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:55.409956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810af98c-4f1a-4362-b725-915f682c0993 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.414400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 807/1624] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:55 2022] PUT /identity/v3/projects/d235718f97954ccb86c48fd996269c96/users/b5582f630d0648e7b159ad7142bc7c70/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:55.415612 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d6ea4-d2bc-433a-95ac-ef3c4a83fdb0 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.425188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810af98c-4f1a-4362-b725-915f682c0993 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:55.510130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87fe98-5d0e-4243-8191-f0f6b7809be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:55.511016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87fe98-5d0e-4243-8191-f0f6b7809be8 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:55.511540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87fe98-5d0e-4243-8191-f0f6b7809be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:55.511923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87fe98-5d0e-4243-8191-f0f6b7809be8 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:55.512971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87fe98-5d0e-4243-8191-f0f6b7809be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:55.516948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87fe98-5d0e-4243-8191-f0f6b7809be8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.540295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d6ea4-d2bc-433a-95ac-ef3c4a83fdb0 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:55.547104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa972d82-e052-465c-9bb2-9aa43a6a5500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:55.547939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa972d82-e052-465c-9bb2-9aa43a6a5500 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:55.548229 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa972d82-e052-465c-9bb2-9aa43a6a5500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:55.548914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa972d82-e052-465c-9bb2-9aa43a6a5500 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:55.549378 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa972d82-e052-465c-9bb2-9aa43a6a5500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:55.552460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa972d82-e052-465c-9bb2-9aa43a6a5500 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.701376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 808/1625] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:55 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:55.702287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa972d82-e052-465c-9bb2-9aa43a6a5500 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.704312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 818/1626] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:43:55 2022] POST /identity/v3/users => generated 292 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:43:55.705026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87fe98-5d0e-4243-8191-f0f6b7809be8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.729242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87fe98-5d0e-4243-8191-f0f6b7809be8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:55.737816 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa972d82-e052-465c-9bb2-9aa43a6a5500 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:55.745410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d148c3-fa96-41d3-8dba-05e51d31d3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:55.746256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdb6d7-c5c7-4c34-9b8d-58ad34ec6441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:55.747355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdb6d7-c5c7-4c34-9b8d-58ad34ec6441 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:55.747355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdb6d7-c5c7-4c34-9b8d-58ad34ec6441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:55.747731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d148c3-fa96-41d3-8dba-05e51d31d3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:55.748112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d148c3-fa96-41d3-8dba-05e51d31d3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:55.748417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d148c3-fa96-41d3-8dba-05e51d31d3eb admin admin] PATH_INFO: `/v3/projects/922290448eab416f9fb6964d4c0ba5e9/users/c65274aee4304a689220b2a78c591139/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:55.752130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdb6d7-c5c7-4c34-9b8d-58ad34ec6441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:55.760604 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdb6d7-c5c7-4c34-9b8d-58ad34ec6441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:55.762558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdb6d7-c5c7-4c34-9b8d-58ad34ec6441 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.769387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d148c3-fa96-41d3-8dba-05e51d31d3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=922290448eab416f9fb6964d4c0ba5e9, user_id=c65274aee4304a689220b2a78c591139, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:55.772048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d148c3-fa96-41d3-8dba-05e51d31d3eb admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.781931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 819/1627] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:55.783226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdb6d7-c5c7-4c34-9b8d-58ad34ec6441 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.796121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdb6d7-c5c7-4c34-9b8d-58ad34ec6441 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:55.803370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de6055-fba3-408f-beb1-1dad225805fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:55.804154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6055-fba3-408f-beb1-1dad225805fe admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:55.804522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6055-fba3-408f-beb1-1dad225805fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:55.804849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6055-fba3-408f-beb1-1dad225805fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:55.805742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de6055-fba3-408f-beb1-1dad225805fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:55.807380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de6055-fba3-408f-beb1-1dad225805fe admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.825547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 820/1628] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:55.826872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de6055-fba3-408f-beb1-1dad225805fe admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.839158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de6055-fba3-408f-beb1-1dad225805fe admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:55.844416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ebcfa-c9b1-4089-a16c-25dc09799c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:55.845083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ebcfa-c9b1-4089-a16c-25dc09799c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:55.845399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ebcfa-c9b1-4089-a16c-25dc09799c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:55.845670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ebcfa-c9b1-4089-a16c-25dc09799c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:55.846413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ebcfa-c9b1-4089-a16c-25dc09799c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:55.847992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ebcfa-c9b1-4089-a16c-25dc09799c28 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.861506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 821/1629] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:55.861506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ebcfa-c9b1-4089-a16c-25dc09799c28 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.877564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ebcfa-c9b1-4089-a16c-25dc09799c28 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:55.886739 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 809/1630] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:55 2022] PUT /identity/v3/projects/922290448eab416f9fb6964d4c0ba5e9/users/c65274aee4304a689220b2a78c591139/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:55.887838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d148c3-fa96-41d3-8dba-05e51d31d3eb admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:55.930421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:55.953846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049fca4-c1a9-4b52-ba5e-a1bdd19b53d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:55.955581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049fca4-c1a9-4b52-ba5e-a1bdd19b53d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:55.956132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049fca4-c1a9-4b52-ba5e-a1bdd19b53d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:55.957248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049fca4-c1a9-4b52-ba5e-a1bdd19b53d9 admin admin] PATH_INFO: `/v3/projects/d235718f97954ccb86c48fd996269c96/users/b5582f630d0648e7b159ad7142bc7c70/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:55.959781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049fca4-c1a9-4b52-ba5e-a1bdd19b53d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d235718f97954ccb86c48fd996269c96, user_id=b5582f630d0648e7b159ad7142bc7c70, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:55.963933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049fca4-c1a9-4b52-ba5e-a1bdd19b53d9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:55.985670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 822/1631] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:55 2022] PUT /identity/v3/projects/d235718f97954ccb86c48fd996269c96/users/b5582f630d0648e7b159ad7142bc7c70/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:55.987311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049fca4-c1a9-4b52-ba5e-a1bdd19b53d9 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:56.055981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d148c3-fa96-41d3-8dba-05e51d31d3eb admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:56.073397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049fca4-c1a9-4b52-ba5e-a1bdd19b53d9 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:56.079682 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bd4fb-56f7-4c77-8511-af291b4212df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:56.081128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bd4fb-56f7-4c77-8511-af291b4212df admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:56.081128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bd4fb-56f7-4c77-8511-af291b4212df admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:56.081455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bd4fb-56f7-4c77-8511-af291b4212df admin admin] PATH_INFO: `/v3/projects/b1856aefd4544e2593528c27303202c2/users/7aedaa48f8374b1fb8003e9c88febe35/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:56.100439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bd4fb-56f7-4c77-8511-af291b4212df admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1856aefd4544e2593528c27303202c2, user_id=7aedaa48f8374b1fb8003e9c88febe35, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:56.104493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bd4fb-56f7-4c77-8511-af291b4212df admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:56.118685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:56.169813 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5935eb-fea4-4790-8ead-c11f1682cd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:56.170733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5935eb-fea4-4790-8ead-c11f1682cd2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:56.171035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5935eb-fea4-4790-8ead-c11f1682cd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:56.171315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5935eb-fea4-4790-8ead-c11f1682cd2b admin admin] PATH_INFO: `/v3/projects/d05c578754344854ab7ae64425bc583a/users/e39a7eeb980543dd9d9c74e55c3a64b6/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:56.188334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5935eb-fea4-4790-8ead-c11f1682cd2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05c578754344854ab7ae64425bc583a, user_id=e39a7eeb980543dd9d9c74e55c3a64b6, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:56.192028 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5935eb-fea4-4790-8ead-c11f1682cd2b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:56.225181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 823/1632] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:55 2022] PUT /identity/v3/projects/b1856aefd4544e2593528c27303202c2/users/7aedaa48f8374b1fb8003e9c88febe35/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:56.226588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bd4fb-56f7-4c77-8511-af291b4212df admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:56.294737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 810/1633] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:55 2022] PUT /identity/v3/projects/d05c578754344854ab7ae64425bc583a/users/e39a7eeb980543dd9d9c74e55c3a64b6/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:56.294737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74b556-3656-43af-8ba9-2d00e394e3f4 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:56.295051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74b556-3656-43af-8ba9-2d00e394e3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:56.295231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74b556-3656-43af-8ba9-2d00e394e3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:56.328252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bd4fb-56f7-4c77-8511-af291b4212df admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:56.332768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-4d74b556-3656-43af-8ba9-2d00e394e3f4 None None] MFA Rules not processed for user `b5582f630d0648e7b159ad7142bc7c70`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:56.334662 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4d74b556-3656-43af-8ba9-2d00e394e3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:56.451300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 811/1634] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:56 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:56.452531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74b556-3656-43af-8ba9-2d00e394e3f4 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:56.462628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b84de4-8c37-48c3-b012-90728043d23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:56.462628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b84de4-8c37-48c3-b012-90728043d23d admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:56.462628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b84de4-8c37-48c3-b012-90728043d23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:56.462628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b84de4-8c37-48c3-b012-90728043d23d admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:56.464357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b84de4-8c37-48c3-b012-90728043d23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:56.466475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b84de4-8c37-48c3-b012-90728043d23d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:56.493659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 824/1635] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:56.493659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b84de4-8c37-48c3-b012-90728043d23d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:56.506836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b84de4-8c37-48c3-b012-90728043d23d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:56.534706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074b004-06d3-4f96-a034-a45f9b788e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:56.535423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074b004-06d3-4f96-a034-a45f9b788e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:56.537038 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074b004-06d3-4f96-a034-a45f9b788e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:56.537038 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074b004-06d3-4f96-a034-a45f9b788e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:56.537038 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074b004-06d3-4f96-a034-a45f9b788e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:56.538408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074b004-06d3-4f96-a034-a45f9b788e40 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:56.559757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 825/1636] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:56.560536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74b556-3656-43af-8ba9-2d00e394e3f4 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:56.560701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074b004-06d3-4f96-a034-a45f9b788e40 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:56.572053 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167419ac-b8b1-44a6-94e1-ead84b14f7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:56.575609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167419ac-b8b1-44a6-94e1-ead84b14f7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:56.575923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167419ac-b8b1-44a6-94e1-ead84b14f7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:56.576209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167419ac-b8b1-44a6-94e1-ead84b14f7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:56.577101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167419ac-b8b1-44a6-94e1-ead84b14f7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:56.579275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167419ac-b8b1-44a6-94e1-ead84b14f7ce admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:56.591753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 812/1637] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:43:56.598857 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167419ac-b8b1-44a6-94e1-ead84b14f7ce admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:56.610416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167419ac-b8b1-44a6-94e1-ead84b14f7ce admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:56.621896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b632e-95d5-4be9-b646-452eff34b9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:56.630981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b632e-95d5-4be9-b646-452eff34b9d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:56.632278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b632e-95d5-4be9-b646-452eff34b9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:56.632569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b632e-95d5-4be9-b646-452eff34b9d0 admin admin] PATH_INFO: `/v3/projects/922290448eab416f9fb6964d4c0ba5e9/users/c65274aee4304a689220b2a78c591139/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:56.638074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b632e-95d5-4be9-b646-452eff34b9d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=922290448eab416f9fb6964d4c0ba5e9, user_id=c65274aee4304a689220b2a78c591139, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:56.645196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b632e-95d5-4be9-b646-452eff34b9d0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:56.667161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 813/1638] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:56 2022] PUT /identity/v3/projects/922290448eab416f9fb6964d4c0ba5e9/users/c65274aee4304a689220b2a78c591139/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:56.670656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b632e-95d5-4be9-b646-452eff34b9d0 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:56.674054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074b004-06d3-4f96-a034-a45f9b788e40 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:56.707334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:56.819743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50b9b6-696c-4af9-9587-d86eff435283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:56.819743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50b9b6-696c-4af9-9587-d86eff435283 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:56.906102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b632e-95d5-4be9-b646-452eff34b9d0 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:56.924837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50b9b6-696c-4af9-9587-d86eff435283 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:56.924837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50b9b6-696c-4af9-9587-d86eff435283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:56.925300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7314-4627-467b-a8ca-2faecc23967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:56.925300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7314-4627-467b-a8ca-2faecc23967e admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:56.925300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7314-4627-467b-a8ca-2faecc23967e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:56.925300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7314-4627-467b-a8ca-2faecc23967e admin admin] PATH_INFO: `/v3/projects/d05c578754344854ab7ae64425bc583a/users/e39a7eeb980543dd9d9c74e55c3a64b6/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:56.925945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7314-4627-467b-a8ca-2faecc23967e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05c578754344854ab7ae64425bc583a, user_id=e39a7eeb980543dd9d9c74e55c3a64b6, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:56.925945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7314-4627-467b-a8ca-2faecc23967e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:56.925945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 814/1639] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:56 2022] PUT /identity/v3/projects/d05c578754344854ab7ae64425bc583a/users/e39a7eeb980543dd9d9c74e55c3a64b6/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:56.925945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7314-4627-467b-a8ca-2faecc23967e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:56.928060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:56.971696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50b9b6-696c-4af9-9587-d86eff435283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:56.973389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50b9b6-696c-4af9-9587-d86eff435283 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:56.986816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 826/1640] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:43:56 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:43:56.988300 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc0d44-bdf1-4bea-8ab7-7a4ee11007fd None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:56.988687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc0d44-bdf1-4bea-8ab7-7a4ee11007fd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:56.989059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc0d44-bdf1-4bea-8ab7-7a4ee11007fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:57.027381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-27bc0d44-bdf1-4bea-8ab7-7a4ee11007fd None None] MFA Rules not processed for user `c65274aee4304a689220b2a78c591139`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:57.031675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-27bc0d44-bdf1-4bea-8ab7-7a4ee11007fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:57.060836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7314-4627-467b-a8ca-2faecc23967e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:57.075212 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37d974-403d-4c86-8342-082ccc101251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:57.075552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37d974-403d-4c86-8342-082ccc101251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:57.075844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37d974-403d-4c86-8342-082ccc101251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:57.076123 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37d974-403d-4c86-8342-082ccc101251 admin admin] PATH_INFO: `/v3/projects/b1856aefd4544e2593528c27303202c2/users/7aedaa48f8374b1fb8003e9c88febe35/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:57.081191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37d974-403d-4c86-8342-082ccc101251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1856aefd4544e2593528c27303202c2, user_id=7aedaa48f8374b1fb8003e9c88febe35, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:57.089216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37d974-403d-4c86-8342-082ccc101251 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:57.113264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 815/1641] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:43:56 2022] PUT /identity/v3/projects/b1856aefd4544e2593528c27303202c2/users/7aedaa48f8374b1fb8003e9c88febe35/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:57.115157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54637b30-316f-4a91-96ae-9c1475b73079 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:57.115157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54637b30-316f-4a91-96ae-9c1475b73079 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:57.481738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54637b30-316f-4a91-96ae-9c1475b73079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:57.483108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 827/1642] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:57.483108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca0302-e306-45ad-870f-a41ecc14cceb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:57.483108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca0302-e306-45ad-870f-a41ecc14cceb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:57.483108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca0302-e306-45ad-870f-a41ecc14cceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:57.484252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-54637b30-316f-4a91-96ae-9c1475b73079 None None] MFA Rules not processed for user `e39a7eeb980543dd9d9c74e55c3a64b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:57.485374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ecca0302-e306-45ad-870f-a41ecc14cceb None None] MFA Rules not processed for user `7aedaa48f8374b1fb8003e9c88febe35`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:57.486177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-54637b30-316f-4a91-96ae-9c1475b73079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:57.486826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ecca0302-e306-45ad-870f-a41ecc14cceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:57.487015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 816/1643] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:57.487015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54637b30-316f-4a91-96ae-9c1475b73079 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:57.487228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 828/1644] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:57.487228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca0302-e306-45ad-870f-a41ecc14cceb None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:57.487506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54637b30-316f-4a91-96ae-9c1475b73079 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:57.487506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd7d36-6248-4a4c-85af-fc1b05e2ff12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:57.487506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd7d36-6248-4a4c-85af-fc1b05e2ff12 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:57.487506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd7d36-6248-4a4c-85af-fc1b05e2ff12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:57.487987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd7d36-6248-4a4c-85af-fc1b05e2ff12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:57.500749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca0302-e306-45ad-870f-a41ecc14cceb None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:57.514235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:57.515833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:57.522829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:57.522829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] PATH_INFO: `/v3/users/78e644cc7e6d46aba520d3454607416b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:57.522829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e644cc7e6d46aba520d3454607416b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:57.522829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: WARNING py.warnings [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:43:57.522829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: warnings.warn(msg) Jul 14 11:43:57.522829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: Jul 14 11:43:57.522829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:57.573040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd7d36-6248-4a4c-85af-fc1b05e2ff12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:57.581841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd7d36-6248-4a4c-85af-fc1b05e2ff12 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:57.595810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 817/1645] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:43:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:43:57.607491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd7d36-6248-4a4c-85af-fc1b05e2ff12 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:57.623864 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd7d36-6248-4a4c-85af-fc1b05e2ff12 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:57.631361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4a52c-cf44-45e7-8f51-877eac3c5549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:57.632098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4a52c-cf44-45e7-8f51-877eac3c5549 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:57.633144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4a52c-cf44-45e7-8f51-877eac3c5549 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:57.633485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4a52c-cf44-45e7-8f51-877eac3c5549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:57.647578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4a52c-cf44-45e7-8f51-877eac3c5549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:57.649770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4a52c-cf44-45e7-8f51-877eac3c5549 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:57.662219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 818/1646] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:43:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:43:57.753092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4a52c-cf44-45e7-8f51-877eac3c5549 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:57.766971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4a52c-cf44-45e7-8f51-877eac3c5549 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:57.772476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0bf19-6eb2-49bb-8775-a5975a8b0b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:57.773534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0bf19-6eb2-49bb-8775-a5975a8b0b7b service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:57.774719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0bf19-6eb2-49bb-8775-a5975a8b0b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:57.775024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0bf19-6eb2-49bb-8775-a5975a8b0b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:57.789061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0bf19-6eb2-49bb-8775-a5975a8b0b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:57.790811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0bf19-6eb2-49bb-8775-a5975a8b0b7b service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:57.815638 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:57.830617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 819/1647] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:43:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:43:57.852080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e644cc7e6d46aba520d3454607416b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:43:57.881779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e644cc7e6d46aba520d3454607416b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:43:57.934364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e644cc7e6d46aba520d3454607416b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:43:58.083585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e644cc7e6d46aba520d3454607416b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:43:58.084251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e644cc7e6d46aba520d3454607416b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:43:58.094166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 829/1648] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:43:57 2022] DELETE /identity/v3/users/78e644cc7e6d46aba520d3454607416b => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:58.097370 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5142e3-5407-4699-a6ab-115811fbe2a0 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:58.097824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5142e3-5407-4699-a6ab-115811fbe2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:58.098237 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5142e3-5407-4699-a6ab-115811fbe2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:58.126500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-6d5142e3-5407-4699-a6ab-115811fbe2a0 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:43:58.129107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6d5142e3-5407-4699-a6ab-115811fbe2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:58.248430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 820/1649] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:43:58.260407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:58.260781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:58.361188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cc6ea-ac8c-4749-b2e8-924a30e9571d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:58.367216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0023a7-c2d3-4708-8ec6-874fb443a1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:58.368371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0023a7-c2d3-4708-8ec6-874fb443a1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:58.368639 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0023a7-c2d3-4708-8ec6-874fb443a1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:58.368893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0023a7-c2d3-4708-8ec6-874fb443a1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:58.387814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0023a7-c2d3-4708-8ec6-874fb443a1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:58.389815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0023a7-c2d3-4708-8ec6-874fb443a1b8 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:58.401439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:58.404973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 830/1650] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:43:58.493863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:59.426258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5142e3-5407-4699-a6ab-115811fbe2a0 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:59.428997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6d5142e3-5407-4699-a6ab-115811fbe2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:59.455618 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5142e3-5407-4699-a6ab-115811fbe2a0 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:59.462397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:59.464407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:59.465580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:59.466003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] PATH_INFO: `/v3/projects/92d73ccf738243868717597ce7e03627` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:59.473495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d73ccf738243868717597ce7e03627)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:59.473495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: WARNING py.warnings [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:43:59.473495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: warnings.warn(msg) Jul 14 11:43:59.473495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: Jul 14 11:43:59.473495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:59.707607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d73ccf738243868717597ce7e03627', 'request_id': 'req-539b6699-d974-4f5a-9d21-8c787585cb10'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:43:59.740677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d73ccf738243868717597ce7e03627', 'request_id': 'req-539b6699-d974-4f5a-9d21-8c787585cb10'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:43:59.749237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0023a7-c2d3-4708-8ec6-874fb443a1b8 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:59.752068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cf0023a7-c2d3-4708-8ec6-874fb443a1b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:59.762519 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d73ccf738243868717597ce7e03627', 'request_id': 'req-539b6699-d974-4f5a-9d21-8c787585cb10'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:43:59.783485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:43:59.785114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0023a7-c2d3-4708-8ec6-874fb443a1b8 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:59.876235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] The token cache is being invalidate because project 92d73ccf738243868717597ce7e03627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:43:59.876841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:43:59.878423 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:43:59.880896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 821/1651] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:43:59 2022] DELETE /identity/v3/projects/92d73ccf738243868717597ce7e03627 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:43:59.885255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:43:59.886824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:43:59.897785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:59.900937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:59.901793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:59.902278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] PATH_INFO: `/v3/users/28b2a44e61794d32bf3b53633657077b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:59.903991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b2a44e61794d32bf3b53633657077b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:59.908282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:43:59.917658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b6699-d974-4f5a-9d21-8c787585cb10 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:43:59.932680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:43:59.933546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:43:59.933829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:43:59.934103 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] PATH_INFO: `/v3/users/b5582f630d0648e7b159ad7142bc7c70` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:43:59.940616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5582f630d0648e7b159ad7142bc7c70)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:43:59.942146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:00.332159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5582f630d0648e7b159ad7142bc7c70'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:00.337343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b2a44e61794d32bf3b53633657077b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:00.341593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5582f630d0648e7b159ad7142bc7c70'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:00.373788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b2a44e61794d32bf3b53633657077b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:00.405124 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5582f630d0648e7b159ad7142bc7c70'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:00.472961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b2a44e61794d32bf3b53633657077b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:00.475679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5582f630d0648e7b159ad7142bc7c70'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:00.615270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5582f630d0648e7b159ad7142bc7c70'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:00.642155 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 822/1652] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:43:59 2022] DELETE /identity/v3/users/b5582f630d0648e7b159ad7142bc7c70 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:00.645505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:00.652671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:00.723468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b2a44e61794d32bf3b53633657077b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:00.724845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b2a44e61794d32bf3b53633657077b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:00.750651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 831/1653] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:43:59 2022] DELETE /identity/v3/users/28b2a44e61794d32bf3b53633657077b => generated 0 bytes in 1002 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:00.760671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:00.773810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:00.843234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:01.024383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac2ba1-549f-42c9-a845-509b1bcec7f7 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:01.039314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:01.040366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:01.040772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:01.042670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] PATH_INFO: `/v3/users/39b601af0566421c9450ea89cc708d16` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:01.044589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] RBAC: Authorizing `identity:delete_user(user_id=39b601af0566421c9450ea89cc708d16)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:01.046479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919f200-53e7-47c4-b69e-3c11878b99fc admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:01.047918 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:01.054868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:01.055866 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:01.056218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:01.057201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] PATH_INFO: `/v3/users/846c97e0d4d84f738df73256856b2498` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:01.073527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=846c97e0d4d84f738df73256856b2498)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:01.075872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:01.312906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39b601af0566421c9450ea89cc708d16'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:01.315392 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39b601af0566421c9450ea89cc708d16'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:01.361615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '846c97e0d4d84f738df73256856b2498'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:01.372375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:01.378919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39b601af0566421c9450ea89cc708d16'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:01.382327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '846c97e0d4d84f738df73256856b2498'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:01.394426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39b601af0566421c9450ea89cc708d16'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:01.421934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '846c97e0d4d84f738df73256856b2498'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:01.479868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7b96c6e9-f5f1-4a70-bff4-3e78aee2b82b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39b601af0566421c9450ea89cc708d16'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:01.495126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 823/1654] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:00 2022] DELETE /identity/v3/users/39b601af0566421c9450ea89cc708d16 => generated 0 bytes in 849 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:01.498332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed193cf-2c29-42a4-9772-a2fd374fa9ed None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:01.498678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed193cf-2c29-42a4-9772-a2fd374fa9ed None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:01.499178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed193cf-2c29-42a4-9772-a2fd374fa9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:01.551649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-7ed193cf-2c29-42a4-9772-a2fd374fa9ed None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:01.556877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7ed193cf-2c29-42a4-9772-a2fd374fa9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:01.598588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '846c97e0d4d84f738df73256856b2498'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:01.599800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-205b81ab-cd74-40e7-b0dc-e79275993ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '846c97e0d4d84f738df73256856b2498'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:01.609107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 832/1655] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:00 2022] DELETE /identity/v3/users/846c97e0d4d84f738df73256856b2498 => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:01.610578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72694b0b-636f-46dc-8241-29aaf19c4691 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:01.610865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72694b0b-636f-46dc-8241-29aaf19c4691 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:01.611241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72694b0b-636f-46dc-8241-29aaf19c4691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:01.686393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-72694b0b-636f-46dc-8241-29aaf19c4691 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:01.688173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-72694b0b-636f-46dc-8241-29aaf19c4691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:01.712168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 824/1656] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:01.713965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8dfe1-1a76-4a2e-9d97-30cf53779f17 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:01.714074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8dfe1-1a76-4a2e-9d97-30cf53779f17 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:01.714324 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8dfe1-1a76-4a2e-9d97-30cf53779f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:01.755771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-98c8dfe1-1a76-4a2e-9d97-30cf53779f17 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:01.757283 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-98c8dfe1-1a76-4a2e-9d97-30cf53779f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:01.869703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 833/1657] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:01.871270 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72694b0b-636f-46dc-8241-29aaf19c4691 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:01.873944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-72694b0b-636f-46dc-8241-29aaf19c4691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:01.931298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 825/1658] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:01.977412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72694b0b-636f-46dc-8241-29aaf19c4691 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:01.985148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a22143-c10d-4365-a2ee-09d267774304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:01.986211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a22143-c10d-4365-a2ee-09d267774304 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:01.986634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a22143-c10d-4365-a2ee-09d267774304 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:01.987046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a22143-c10d-4365-a2ee-09d267774304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:01.990704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8dfe1-1a76-4a2e-9d97-30cf53779f17 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:01.992509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-98c8dfe1-1a76-4a2e-9d97-30cf53779f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:02.008652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8dfe1-1a76-4a2e-9d97-30cf53779f17 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:02.015630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a22143-c10d-4365-a2ee-09d267774304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:02.017092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a22143-c10d-4365-a2ee-09d267774304 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:02.021845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3fbe8-97d3-4418-98b4-2f2e87c51f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:02.022516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3fbe8-97d3-4418-98b4-2f2e87c51f82 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:02.023565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3fbe8-97d3-4418-98b4-2f2e87c51f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:02.023565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3fbe8-97d3-4418-98b4-2f2e87c51f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:02.042667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 834/1659] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:02.043762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a22143-c10d-4365-a2ee-09d267774304 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:02.045396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3fbe8-97d3-4418-98b4-2f2e87c51f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:02.056241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3fbe8-97d3-4418-98b4-2f2e87c51f82 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:02.068891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a22143-c10d-4365-a2ee-09d267774304 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:02.082193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 826/1660] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:02.082696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b5c75-1381-47fa-aaab-e293b3f9125d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:02.082696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b5c75-1381-47fa-aaab-e293b3f9125d service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:02.082696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b5c75-1381-47fa-aaab-e293b3f9125d service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:02.082696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b5c75-1381-47fa-aaab-e293b3f9125d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:02.098528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b5c75-1381-47fa-aaab-e293b3f9125d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:02.100445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b5c75-1381-47fa-aaab-e293b3f9125d service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:02.114744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 835/1661] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:02.415671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:02.732891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3fbe8-97d3-4418-98b4-2f2e87c51f82 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:02.734403 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b5b3fbe8-97d3-4418-98b4-2f2e87c51f82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:02.761471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3fbe8-97d3-4418-98b4-2f2e87c51f82 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:02.779328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:02.780355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:02.780639 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:02.780913 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] PATH_INFO: `/v3/projects/d235718f97954ccb86c48fd996269c96` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:02.790225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d235718f97954ccb86c48fd996269c96)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:02.792470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:02.974523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd235718f97954ccb86c48fd996269c96', 'request_id': 'req-52d66680-e11c-40b1-93a8-bdebb89a24e5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:02.990639 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd235718f97954ccb86c48fd996269c96', 'request_id': 'req-52d66680-e11c-40b1-93a8-bdebb89a24e5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:03.004984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:03.011458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd235718f97954ccb86c48fd996269c96', 'request_id': 'req-52d66680-e11c-40b1-93a8-bdebb89a24e5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:03.127913 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] The token cache is being invalidate because project d235718f97954ccb86c48fd996269c96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:03.128797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:03.131231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-52d66680-e11c-40b1-93a8-bdebb89a24e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:03.138819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 827/1662] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:02 2022] DELETE /identity/v3/projects/d235718f97954ccb86c48fd996269c96 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:03.145403 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88e886-c91d-4a2c-abb1-6db1dbeefab5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:03.147006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88e886-c91d-4a2c-abb1-6db1dbeefab5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:03.147506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88e886-c91d-4a2c-abb1-6db1dbeefab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:03.178728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ee88e886-c91d-4a2c-abb1-6db1dbeefab5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:03.180215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ee88e886-c91d-4a2c-abb1-6db1dbeefab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:03.340241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 836/1663] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:03.347794 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b582e65-1b8b-4b0c-a802-70bf331607c3 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:03.348086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b582e65-1b8b-4b0c-a802-70bf331607c3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:03.348859 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b582e65-1b8b-4b0c-a802-70bf331607c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:03.369685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88e886-c91d-4a2c-abb1-6db1dbeefab5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:03.373627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ee88e886-c91d-4a2c-abb1-6db1dbeefab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:03.405898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88e886-c91d-4a2c-abb1-6db1dbeefab5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:03.412906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-7b582e65-1b8b-4b0c-a802-70bf331607c3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:03.419320 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7b582e65-1b8b-4b0c-a802-70bf331607c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:03.420905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:03.422154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:03.422435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:03.422970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] PATH_INFO: `/v3/projects/b2c5dbb0443e40109e2e82c19464128c` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:03.425360 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c5dbb0443e40109e2e82c19464128c)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:03.427126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:03.443201 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 828/1664] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:03.445347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b582e65-1b8b-4b0c-a802-70bf331607c3 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:03.471886 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b582e65-1b8b-4b0c-a802-70bf331607c3 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:03.486319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f12a75-8684-46fc-9bc6-4b7b859d63f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:03.487026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f12a75-8684-46fc-9bc6-4b7b859d63f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:03.487344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f12a75-8684-46fc-9bc6-4b7b859d63f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:03.487582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f12a75-8684-46fc-9bc6-4b7b859d63f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:03.491970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f12a75-8684-46fc-9bc6-4b7b859d63f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:03.492317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f12a75-8684-46fc-9bc6-4b7b859d63f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:03.498215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f12a75-8684-46fc-9bc6-4b7b859d63f9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:03.527969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 829/1665] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:03.531888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f12a75-8684-46fc-9bc6-4b7b859d63f9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:03.553834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f12a75-8684-46fc-9bc6-4b7b859d63f9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:03.568852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60581bb-5d2a-4164-9c65-335a9a48327d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:03.570119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60581bb-5d2a-4164-9c65-335a9a48327d admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:03.571660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60581bb-5d2a-4164-9c65-335a9a48327d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:03.572801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60581bb-5d2a-4164-9c65-335a9a48327d admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:03.574120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60581bb-5d2a-4164-9c65-335a9a48327d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:03.580844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60581bb-5d2a-4164-9c65-335a9a48327d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:03.591730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c5dbb0443e40109e2e82c19464128c', 'request_id': 'req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:03.723734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c5dbb0443e40109e2e82c19464128c', 'request_id': 'req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:03.732187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c5dbb0443e40109e2e82c19464128c', 'request_id': 'req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:03.765714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] The token cache is being invalidate because project b2c5dbb0443e40109e2e82c19464128c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:03.766113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 830/1666] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:03 2022] POST /identity/v3/projects => generated 407 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:03.766730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:03.767062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60581bb-5d2a-4164-9c65-335a9a48327d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:03.768118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:03.776163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 837/1667] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:03 2022] DELETE /identity/v3/projects/b2c5dbb0443e40109e2e82c19464128c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:03.778418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:03.780585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:03.928520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60581bb-5d2a-4164-9c65-335a9a48327d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:03.935245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9caa6-ba6e-40a1-8da2-9c02477b2e42 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:03.941557 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b193565-f302-43fe-8b8d-406a60d2529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:03.942259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b193565-f302-43fe-8b8d-406a60d2529e admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:03.942594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b193565-f302-43fe-8b8d-406a60d2529e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:03.942876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b193565-f302-43fe-8b8d-406a60d2529e admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:03.945136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b193565-f302-43fe-8b8d-406a60d2529e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:03.945599 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b193565-f302-43fe-8b8d-406a60d2529e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:03.964258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:03.967653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:03.969815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:03.970336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] PATH_INFO: `/v3/users/e39a7eeb980543dd9d9c74e55c3a64b6` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:03.972820 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39a7eeb980543dd9d9c74e55c3a64b6)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:03.979615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:04.034142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:04.100741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 831/1668] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:03 2022] POST /identity/v3/users => generated 304 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:04.108960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b193565-f302-43fe-8b8d-406a60d2529e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:04.110854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7b193565-f302-43fe-8b8d-406a60d2529e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:04.133928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b193565-f302-43fe-8b8d-406a60d2529e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:04.148927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfa6ed-e89e-4234-a878-1e9aad8f1633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:04.149916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfa6ed-e89e-4234-a878-1e9aad8f1633 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:04.150888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfa6ed-e89e-4234-a878-1e9aad8f1633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:04.151344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfa6ed-e89e-4234-a878-1e9aad8f1633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:04.152238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfa6ed-e89e-4234-a878-1e9aad8f1633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:04.157236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfa6ed-e89e-4234-a878-1e9aad8f1633 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:04.182377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 832/1669] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:44:04.186448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfa6ed-e89e-4234-a878-1e9aad8f1633 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:04.191046 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4ddfa6ed-e89e-4234-a878-1e9aad8f1633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:04.213078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfa6ed-e89e-4234-a878-1e9aad8f1633 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:04.218765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:04.219894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:04.220314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:04.220733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] PATH_INFO: `/v3/projects/00a4eb6dbc8b47d39835a10a93e586e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:04.222165 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a4eb6dbc8b47d39835a10a93e586e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:04.224320 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:04.278151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39a7eeb980543dd9d9c74e55c3a64b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:04.293660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39a7eeb980543dd9d9c74e55c3a64b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:04.325571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39a7eeb980543dd9d9c74e55c3a64b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:04.403299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a4eb6dbc8b47d39835a10a93e586e9', 'request_id': 'req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:04.424439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a4eb6dbc8b47d39835a10a93e586e9', 'request_id': 'req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:04.426429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39a7eeb980543dd9d9c74e55c3a64b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:04.427177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39a7eeb980543dd9d9c74e55c3a64b6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:04.448944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 838/1670] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:03 2022] DELETE /identity/v3/users/e39a7eeb980543dd9d9c74e55c3a64b6 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:04.448944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:04.449076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a4eb6dbc8b47d39835a10a93e586e9', 'request_id': 'req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:04.449145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:04.576055 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] The token cache is being invalidate because project 00a4eb6dbc8b47d39835a10a93e586e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:04.580487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:04.580487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:04.580487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 833/1671] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:04 2022] DELETE /identity/v3/projects/00a4eb6dbc8b47d39835a10a93e586e9 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:04.583684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:04.594635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:04.612631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15f254-e972-44bb-8077-836f874484ad admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:04.625511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c46fbc-f063-4bc0-a0df-1b354bca6298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:04.626352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c46fbc-f063-4bc0-a0df-1b354bca6298 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:04.631046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c46fbc-f063-4bc0-a0df-1b354bca6298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:04.631046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c46fbc-f063-4bc0-a0df-1b354bca6298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:04.631046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c46fbc-f063-4bc0-a0df-1b354bca6298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:04.631046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c46fbc-f063-4bc0-a0df-1b354bca6298 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:04.671695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 839/1672] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:04.686694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c46fbc-f063-4bc0-a0df-1b354bca6298 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:04.688550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-07c46fbc-f063-4bc0-a0df-1b354bca6298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:04.732034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c46fbc-f063-4bc0-a0df-1b354bca6298 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:04.750337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18e5e8-ceff-4c08-90e9-09069839ed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:04.751256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e5e8-ceff-4c08-90e9-09069839ed28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:04.751536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e5e8-ceff-4c08-90e9-09069839ed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:04.751812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e5e8-ceff-4c08-90e9-09069839ed28 admin admin] PATH_INFO: `/v3/projects/35423dbae50241b1b07df9d84b849588/users/9c8674cd6707429e84db02796f3c0db9/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:04.783916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18e5e8-ceff-4c08-90e9-09069839ed28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35423dbae50241b1b07df9d84b849588, user_id=9c8674cd6707429e84db02796f3c0db9, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:04.786629 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18e5e8-ceff-4c08-90e9-09069839ed28 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:04.804927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4a4b1-634e-48c8-8b0c-2219a1a6bbe9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:04.818349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:04.819568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:04.819855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:04.820132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] PATH_INFO: `/v3/users/c65274aee4304a689220b2a78c591139` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:04.821914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c65274aee4304a689220b2a78c591139)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:04.824154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:04.901907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 840/1673] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:04 2022] PUT /identity/v3/projects/35423dbae50241b1b07df9d84b849588/users/9c8674cd6707429e84db02796f3c0db9/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:04.903133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18e5e8-ceff-4c08-90e9-09069839ed28 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:04.906265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6e18e5e8-ceff-4c08-90e9-09069839ed28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:05.065603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18e5e8-ceff-4c08-90e9-09069839ed28 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:05.075660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:05.076908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:05.078511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:05.079256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] PATH_INFO: `/v3/projects/5eb85e33b57d4660846c2dcd31bce190` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:05.083155 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eb85e33b57d4660846c2dcd31bce190)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:05.086891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:05.166581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:DELETE=9,keystone:SELECT=85,keystone:INSERT=6 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:05.175892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c65274aee4304a689220b2a78c591139'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.176600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c65274aee4304a689220b2a78c591139'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.193829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c65274aee4304a689220b2a78c591139'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.228028 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c65274aee4304a689220b2a78c591139'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.235059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb85e33b57d4660846c2dcd31bce190', 'request_id': 'req-c4dd249d-4395-4bc4-b24a-5de25e097deb'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.355858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb85e33b57d4660846c2dcd31bce190', 'request_id': 'req-c4dd249d-4395-4bc4-b24a-5de25e097deb'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.365984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb85e33b57d4660846c2dcd31bce190', 'request_id': 'req-c4dd249d-4395-4bc4-b24a-5de25e097deb'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.402234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] The token cache is being invalidate because project 5eb85e33b57d4660846c2dcd31bce190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:05.404048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.405793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c4dd249d-4395-4bc4-b24a-5de25e097deb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.416864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 841/1674] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:04 2022] DELETE /identity/v3/projects/5eb85e33b57d4660846c2dcd31bce190 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:05.421846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2285fb-4e80-4196-95d8-0d640e7ab3d8 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:05.422388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2285fb-4e80-4196-95d8-0d640e7ab3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:05.423545 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2285fb-4e80-4196-95d8-0d640e7ab3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:05.431020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c65274aee4304a689220b2a78c591139'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.441068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 834/1675] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:04 2022] DELETE /identity/v3/users/c65274aee4304a689220b2a78c591139 => generated 0 bytes in 858 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:05.445099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:05.450288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:05.508958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-bb2285fb-4e80-4196-95d8-0d640e7ab3d8 None None] MFA Rules not processed for user `9c8674cd6707429e84db02796f3c0db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:05.515504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bb2285fb-4e80-4196-95d8-0d640e7ab3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:05.626142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158d202-490c-4db7-870d-02f689fd5df3 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:05.643271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:05.644183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:05.644475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:05.646028 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] PATH_INFO: `/v3/users/7aedaa48f8374b1fb8003e9c88febe35` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:05.647996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aedaa48f8374b1fb8003e9c88febe35)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:05.654290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:05.737959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 842/1676] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:05 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:05.738638 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2285fb-4e80-4196-95d8-0d640e7ab3d8 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:05.744803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bb2285fb-4e80-4196-95d8-0d640e7ab3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:05.774194 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2285fb-4e80-4196-95d8-0d640e7ab3d8 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:05.786225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb7a76-4b82-4011-a55b-f3e57a05d2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:05.787330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb7a76-4b82-4011-a55b-f3e57a05d2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:05.787698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb7a76-4b82-4011-a55b-f3e57a05d2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:05.788180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb7a76-4b82-4011-a55b-f3e57a05d2c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:05.789465 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb7a76-4b82-4011-a55b-f3e57a05d2c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:05.799971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb7a76-4b82-4011-a55b-f3e57a05d2c5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:05.883408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aedaa48f8374b1fb8003e9c88febe35'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.887400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aedaa48f8374b1fb8003e9c88febe35'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.923084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aedaa48f8374b1fb8003e9c88febe35'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.944109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aedaa48f8374b1fb8003e9c88febe35'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:05.991326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 843/1677] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:05 2022] POST /identity/v3/projects => generated 409 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:05.996029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb7a76-4b82-4011-a55b-f3e57a05d2c5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:05.997921 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-efeb7a76-4b82-4011-a55b-f3e57a05d2c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:06.057818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aedaa48f8374b1fb8003e9c88febe35'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:06.071674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 835/1678] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:05 2022] DELETE /identity/v3/users/7aedaa48f8374b1fb8003e9c88febe35 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:06.087388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:06.148106 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb7a76-4b82-4011-a55b-f3e57a05d2c5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:06.243806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160b93-fe92-48be-b070-af07b97627b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:06.245273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160b93-fe92-48be-b070-af07b97627b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:06.245564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160b93-fe92-48be-b070-af07b97627b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:06.245859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160b93-fe92-48be-b070-af07b97627b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:06.246653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46160b93-fe92-48be-b070-af07b97627b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:06.252950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46160b93-fe92-48be-b070-af07b97627b9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:06.628167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 844/1679] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:05 2022] POST /identity/v3/users => generated 311 bytes in 630 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:06.629242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:06.665308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e24cde-9a36-43c4-b2b7-aee093786974 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:06.696234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c81dd3-02d0-4b47-ab8a-100d3b867991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:06.698549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c81dd3-02d0-4b47-ab8a-100d3b867991 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:06.698888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c81dd3-02d0-4b47-ab8a-100d3b867991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:06.699167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c81dd3-02d0-4b47-ab8a-100d3b867991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:06.700623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c81dd3-02d0-4b47-ab8a-100d3b867991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:06.702626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c81dd3-02d0-4b47-ab8a-100d3b867991 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:06.731058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 836/1680] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:06.739460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160b93-fe92-48be-b070-af07b97627b9 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:06.758803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160b93-fe92-48be-b070-af07b97627b9 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:06.769678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751c026-cc54-4a32-bbcc-dbed29a7a798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:06.771461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751c026-cc54-4a32-bbcc-dbed29a7a798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:06.771756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751c026-cc54-4a32-bbcc-dbed29a7a798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:06.774127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751c026-cc54-4a32-bbcc-dbed29a7a798 admin admin] PATH_INFO: `/v3/projects/0abd60efd2e04d66bc5ebc99341f9c8d/users/9190fcc540be4dc7abafe41ddf21fce2/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:06.823703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751c026-cc54-4a32-bbcc-dbed29a7a798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0abd60efd2e04d66bc5ebc99341f9c8d, user_id=9190fcc540be4dc7abafe41ddf21fce2, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:06.839408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751c026-cc54-4a32-bbcc-dbed29a7a798 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:06.889452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c81dd3-02d0-4b47-ab8a-100d3b867991 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:06.892561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-46c81dd3-02d0-4b47-ab8a-100d3b867991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:06.930566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c81dd3-02d0-4b47-ab8a-100d3b867991 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:06.939314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:06.940203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:06.940494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:06.940770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] PATH_INFO: `/v3/projects/d05c578754344854ab7ae64425bc583a` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:06.942200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05c578754344854ab7ae64425bc583a)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:06.944158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:06.976211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 845/1681] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:06 2022] PUT /identity/v3/projects/0abd60efd2e04d66bc5ebc99341f9c8d/users/9190fcc540be4dc7abafe41ddf21fce2/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:06.976211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751c026-cc54-4a32-bbcc-dbed29a7a798 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:07.109871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751c026-cc54-4a32-bbcc-dbed29a7a798 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:07.115809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54551a5-8911-4734-bf41-280af156dc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:07.117115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54551a5-8911-4734-bf41-280af156dc17 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:07.117615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54551a5-8911-4734-bf41-280af156dc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:07.118733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54551a5-8911-4734-bf41-280af156dc17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:07.119719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54551a5-8911-4734-bf41-280af156dc17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:07.121863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54551a5-8911-4734-bf41-280af156dc17 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:07.162894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 846/1682] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:07.167569 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54551a5-8911-4734-bf41-280af156dc17 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:07.173385 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05c578754344854ab7ae64425bc583a', 'request_id': 'req-aab4099d-86a9-4752-984c-0467e021df50'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:07.188831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54551a5-8911-4734-bf41-280af156dc17 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:07.201435 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05c578754344854ab7ae64425bc583a', 'request_id': 'req-aab4099d-86a9-4752-984c-0467e021df50'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:07.207376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:07.216916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05c578754344854ab7ae64425bc583a', 'request_id': 'req-aab4099d-86a9-4752-984c-0467e021df50'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:07.336858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] The token cache is being invalidate because project d05c578754344854ab7ae64425bc583a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:07.342731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:07.342855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727df13-2c74-4b0a-ab01-a462c49621e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:07.342855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727df13-2c74-4b0a-ab01-a462c49621e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:07.342855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727df13-2c74-4b0a-ab01-a462c49621e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:07.342855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727df13-2c74-4b0a-ab01-a462c49621e3 admin admin] PATH_INFO: `/v3/projects/0abd60efd2e04d66bc5ebc99341f9c8d/users/9190fcc540be4dc7abafe41ddf21fce2/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:07.343146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aab4099d-86a9-4752-984c-0467e021df50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:07.343146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 837/1683] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:06 2022] DELETE /identity/v3/projects/d05c578754344854ab7ae64425bc583a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:07.344489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbf359-bb68-41ee-b820-b20fd4ade545 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:07.344797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbf359-bb68-41ee-b820-b20fd4ade545 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:07.345288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbf359-bb68-41ee-b820-b20fd4ade545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:07.347871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727df13-2c74-4b0a-ab01-a462c49621e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0abd60efd2e04d66bc5ebc99341f9c8d, user_id=9190fcc540be4dc7abafe41ddf21fce2, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:07.357864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727df13-2c74-4b0a-ab01-a462c49621e3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:07.393237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 847/1684] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:07 2022] PUT /identity/v3/projects/0abd60efd2e04d66bc5ebc99341f9c8d/users/9190fcc540be4dc7abafe41ddf21fce2/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:07.395844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db407896-d22e-4263-b5c4-e31689bec732 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:07.399222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db407896-d22e-4263-b5c4-e31689bec732 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:07.401760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db407896-d22e-4263-b5c4-e31689bec732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:07.439809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-79dbf359-bb68-41ee-b820-b20fd4ade545 None None] MFA Rules not processed for user `760c5416ab5c44a59eb03971c2252725`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:07.444274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-79dbf359-bb68-41ee-b820-b20fd4ade545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:07.464366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-db407896-d22e-4263-b5c4-e31689bec732 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:07.467231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-db407896-d22e-4263-b5c4-e31689bec732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:07.695989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 848/1685] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:07.698752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b03912-3461-461e-bd4e-93a65c8bb2bb None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:07.699266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b03912-3461-461e-bd4e-93a65c8bb2bb None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:07.699867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b03912-3461-461e-bd4e-93a65c8bb2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:07.740062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 838/1686] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:07.740062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbf359-bb68-41ee-b820-b20fd4ade545 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:07.742363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-79dbf359-bb68-41ee-b820-b20fd4ade545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:07.751187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e6b03912-3461-461e-bd4e-93a65c8bb2bb None None] MFA Rules not processed for user `9190fcc540be4dc7abafe41ddf21fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:07.753036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e6b03912-3461-461e-bd4e-93a65c8bb2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:07.800947 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbf359-bb68-41ee-b820-b20fd4ade545 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:07.825638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:07.825638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:07.825638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:07.825638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] PATH_INFO: `/v3/projects/922290448eab416f9fb6964d4c0ba5e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:07.828460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] RBAC: Authorizing `identity:delete_project(project_id=922290448eab416f9fb6964d4c0ba5e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:07.840101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:07.948081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 849/1687] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:07 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:07.949957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871ccbe-ba88-4cce-a8b2-de4f9abe9f8e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:07.950489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871ccbe-ba88-4cce-a8b2-de4f9abe9f8e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:07.950908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871ccbe-ba88-4cce-a8b2-de4f9abe9f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:07.973298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b871ccbe-ba88-4cce-a8b2-de4f9abe9f8e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:07.974746 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b871ccbe-ba88-4cce-a8b2-de4f9abe9f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:07.995079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 850/1688] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:07.997058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51451-c82d-4c69-9217-6bfa6159049a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:07.997388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51451-c82d-4c69-9217-6bfa6159049a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:07.998117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51451-c82d-4c69-9217-6bfa6159049a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:08.050744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2cd51451-c82d-4c69-9217-6bfa6159049a None None] MFA Rules not processed for user `760c5416ab5c44a59eb03971c2252725`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:08.053498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2cd51451-c82d-4c69-9217-6bfa6159049a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:08.069020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 851/1689] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:08.070004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd51451-c82d-4c69-9217-6bfa6159049a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:08.072098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2cd51451-c82d-4c69-9217-6bfa6159049a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:08.100120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '922290448eab416f9fb6964d4c0ba5e9', 'request_id': 'req-92a7240d-7243-4694-8744-5620b342995c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:08.120891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '922290448eab416f9fb6964d4c0ba5e9', 'request_id': 'req-92a7240d-7243-4694-8744-5620b342995c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:08.144959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '922290448eab416f9fb6964d4c0ba5e9', 'request_id': 'req-92a7240d-7243-4694-8744-5620b342995c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:08.145168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:08.219036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd51451-c82d-4c69-9217-6bfa6159049a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:08.232615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] The token cache is being invalidate because project 922290448eab416f9fb6964d4c0ba5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:08.232948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:08.234299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:08.240511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 839/1690] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:07 2022] DELETE /identity/v3/projects/922290448eab416f9fb6964d4c0ba5e9 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:08.240511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:08.243255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:08.311399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5af08-dcd9-4f86-b56a-34b3b29a498f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:08.311399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5af08-dcd9-4f86-b56a-34b3b29a498f service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:08.311657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5af08-dcd9-4f86-b56a-34b3b29a498f service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:08.312382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5af08-dcd9-4f86-b56a-34b3b29a498f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:08.315030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-07f5af08-dcd9-4f86-b56a-34b3b29a498f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:08.435844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7240d-7243-4694-8744-5620b342995c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:08.445060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5af08-dcd9-4f86-b56a-34b3b29a498f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:08.446739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5af08-dcd9-4f86-b56a-34b3b29a498f service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:08.450960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db3e0b-38e5-4283-923f-37b8f0b9908d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:08.451651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db3e0b-38e5-4283-923f-37b8f0b9908d admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:08.451932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db3e0b-38e5-4283-923f-37b8f0b9908d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:08.452207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db3e0b-38e5-4283-923f-37b8f0b9908d admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:08.453209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db3e0b-38e5-4283-923f-37b8f0b9908d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:08.453313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db3e0b-38e5-4283-923f-37b8f0b9908d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:08.454918 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db3e0b-38e5-4283-923f-37b8f0b9908d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:08.476782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 852/1691] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:08 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:08.481533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9a021-ed60-432b-b53e-987589707798 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:08.497133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9a021-ed60-432b-b53e-987589707798 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:08.497133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9a021-ed60-432b-b53e-987589707798 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:08.497133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 853/1692] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jul 14 11:44:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:44:08.497133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcae09a-d475-4186-8b1e-4eb71b8e9bb3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:08.497133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcae09a-d475-4186-8b1e-4eb71b8e9bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:08.497133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcae09a-d475-4186-8b1e-4eb71b8e9bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:08.498150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 840/1693] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:08.498150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9a9bc-c7f9-42df-a90d-24489b33230b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:08.498150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9a9bc-c7f9-42df-a90d-24489b33230b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:08.498150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9a9bc-c7f9-42df-a90d-24489b33230b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:08.536352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e7f9a9bc-c7f9-42df-a90d-24489b33230b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:08.536756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2dcae09a-d475-4186-8b1e-4eb71b8e9bb3 None None] MFA Rules not processed for user `246f07024efa444da74f2b898f426b46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:08.537601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e7f9a9bc-c7f9-42df-a90d-24489b33230b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:08.539823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2dcae09a-d475-4186-8b1e-4eb71b8e9bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:08.570844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 841/1694] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:08.572898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb43c84-635a-4c19-a806-563eeec6312f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:08.573018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb43c84-635a-4c19-a806-563eeec6312f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:08.582504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb43c84-635a-4c19-a806-563eeec6312f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:08.678255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-2bb43c84-635a-4c19-a806-563eeec6312f None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:08.688989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2bb43c84-635a-4c19-a806-563eeec6312f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:08.715661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 854/1695] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:08.717610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcae09a-d475-4186-8b1e-4eb71b8e9bb3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:08.747086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcae09a-d475-4186-8b1e-4eb71b8e9bb3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:08.761838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bb2de-60e8-454d-86be-00a475bacf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:08.764359 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bb2de-60e8-454d-86be-00a475bacf61 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:08.764651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bb2de-60e8-454d-86be-00a475bacf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:08.764929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bb2de-60e8-454d-86be-00a475bacf61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:08.765106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:08.767897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bb2de-60e8-454d-86be-00a475bacf61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:08.772738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bb2de-60e8-454d-86be-00a475bacf61 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:08.844589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 842/1696] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jul 14 11:44:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:08.858858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac7126-09a0-4378-bfc9-2edc4356fc70 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:08.859150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac7126-09a0-4378-bfc9-2edc4356fc70 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:08.859432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac7126-09a0-4378-bfc9-2edc4356fc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:08.898798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-adac7126-09a0-4378-bfc9-2edc4356fc70 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:08.902478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 855/1697] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:08 2022] POST /identity/v3/projects => generated 357 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:08.903289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-adac7126-09a0-4378-bfc9-2edc4356fc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:08.903607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dba36-4546-4808-b424-4d1b8ef34548 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:08.903904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dba36-4546-4808-b424-4d1b8ef34548 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:08.906995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dba36-4546-4808-b424-4d1b8ef34548 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:08.910750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 856/1698] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jul 14 11:44:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:44:08.914113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dba36-4546-4808-b424-4d1b8ef34548 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:08.916635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-528dba36-4546-4808-b424-4d1b8ef34548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:09.031225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 843/1699] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:09.033104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac7126-09a0-4378-bfc9-2edc4356fc70 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:09.105517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dba36-4546-4808-b424-4d1b8ef34548 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:09.121326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:09.122200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:09.122479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:09.122749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] PATH_INFO: `/v3/projects/b1856aefd4544e2593528c27303202c2` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:09.124004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1856aefd4544e2593528c27303202c2)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:09.125974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:09.223095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac7126-09a0-4378-bfc9-2edc4356fc70 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:09.239480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe565496-9159-408d-b84c-0e1d3eaa7ed1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:09.240227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe565496-9159-408d-b84c-0e1d3eaa7ed1 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:09.245518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe565496-9159-408d-b84c-0e1d3eaa7ed1 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:09.245518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe565496-9159-408d-b84c-0e1d3eaa7ed1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:09.245518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-fe565496-9159-408d-b84c-0e1d3eaa7ed1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:09.302892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1856aefd4544e2593528c27303202c2', 'request_id': 'req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:09.398167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1856aefd4544e2593528c27303202c2', 'request_id': 'req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:09.406253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1856aefd4544e2593528c27303202c2', 'request_id': 'req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:09.422957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe565496-9159-408d-b84c-0e1d3eaa7ed1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:09.424589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe565496-9159-408d-b84c-0e1d3eaa7ed1 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:09.426730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] The token cache is being invalidate because project b1856aefd4544e2593528c27303202c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:09.427133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:09.427931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:09.436832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 857/1700] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:08 2022] DELETE /identity/v3/projects/b1856aefd4544e2593528c27303202c2 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:09.438500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:09.440721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:09.452792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 844/1701] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:09.454329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c6863-655f-472a-93c3-0a1132faf135 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:09.454617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c6863-655f-472a-93c3-0a1132faf135 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:09.455267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c6863-655f-472a-93c3-0a1132faf135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:09.498885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-533c6863-655f-472a-93c3-0a1132faf135 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:09.502266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-533c6863-655f-472a-93c3-0a1132faf135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:09.528816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2043a-b1f4-48ef-a8ae-8f4b4f2d9f63 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:09.535044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0ad5d-180d-43f0-9b77-e8bbc550d9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:09.536033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0ad5d-180d-43f0-9b77-e8bbc550d9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:09.543487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0ad5d-180d-43f0-9b77-e8bbc550d9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:09.543487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0ad5d-180d-43f0-9b77-e8bbc550d9b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:09.543487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0ad5d-180d-43f0-9b77-e8bbc550d9b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:09.543487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0ad5d-180d-43f0-9b77-e8bbc550d9b2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:09.613646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 845/1702] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jul 14 11:44:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:09.616688 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c6863-655f-472a-93c3-0a1132faf135 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:09.616688 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-533c6863-655f-472a-93c3-0a1132faf135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:09.633869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c6863-655f-472a-93c3-0a1132faf135 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:09.639645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b92b5-bc39-4db4-af4d-7c064f00c03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:09.640536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b92b5-bc39-4db4-af4d-7c064f00c03e admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:09.640679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b92b5-bc39-4db4-af4d-7c064f00c03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:09.640954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b92b5-bc39-4db4-af4d-7c064f00c03e admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:09.641736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b92b5-bc39-4db4-af4d-7c064f00c03e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:09.642175 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b92b5-bc39-4db4-af4d-7c064f00c03e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:09.644052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b92b5-bc39-4db4-af4d-7c064f00c03e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:09.655549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 858/1703] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:09 2022] POST /identity/v3/users => generated 279 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:09.661705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0ad5d-180d-43f0-9b77-e8bbc550d9b2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:09.664296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 846/1704] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:09.665750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734466f-3773-4078-b2c0-624f998983cf None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:09.666109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734466f-3773-4078-b2c0-624f998983cf None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:09.666544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734466f-3773-4078-b2c0-624f998983cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:09.669729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-79d0ad5d-180d-43f0-9b77-e8bbc550d9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:09.703236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-7734466f-3773-4078-b2c0-624f998983cf None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:09.704656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7734466f-3773-4078-b2c0-624f998983cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:09.717107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 847/1705] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:09.720168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734466f-3773-4078-b2c0-624f998983cf None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:09.746238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:09.752900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734466f-3773-4078-b2c0-624f998983cf None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:09.765802 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4665681-ad6a-4376-91ee-bdff9b3b8411 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:09.767832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4665681-ad6a-4376-91ee-bdff9b3b8411 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:09.767832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4665681-ad6a-4376-91ee-bdff9b3b8411 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:09.767832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4665681-ad6a-4376-91ee-bdff9b3b8411 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:09.777957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d4665681-ad6a-4376-91ee-bdff9b3b8411 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:09.878931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0ad5d-180d-43f0-9b77-e8bbc550d9b2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:09.902089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dde5aa-54ef-4ce4-8375-8ef5c0647d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:09.903488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde5aa-54ef-4ce4-8375-8ef5c0647d30 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:09.903874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde5aa-54ef-4ce4-8375-8ef5c0647d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:09.904255 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde5aa-54ef-4ce4-8375-8ef5c0647d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:09.907884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f0dde5aa-54ef-4ce4-8375-8ef5c0647d30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:09.935587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4665681-ad6a-4376-91ee-bdff9b3b8411 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:09.937858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4665681-ad6a-4376-91ee-bdff9b3b8411 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:09.974413 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 848/1706] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:09.975404 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4665681-ad6a-4376-91ee-bdff9b3b8411 service nova] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:10.003652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4665681-ad6a-4376-91ee-bdff9b3b8411 service nova] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:10.018536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe795f1-4b89-4dec-b8d4-6397716ed357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:10.019241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe795f1-4b89-4dec-b8d4-6397716ed357 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:10.019523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe795f1-4b89-4dec-b8d4-6397716ed357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:10.019866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe795f1-4b89-4dec-b8d4-6397716ed357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:10.020527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe795f1-4b89-4dec-b8d4-6397716ed357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:10.022904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe795f1-4b89-4dec-b8d4-6397716ed357 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:10.030126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dde5aa-54ef-4ce4-8375-8ef5c0647d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:10.033854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dde5aa-54ef-4ce4-8375-8ef5c0647d30 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:10.056820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 849/1707] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:10.057003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 859/1708] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:09 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:10.057099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe795f1-4b89-4dec-b8d4-6397716ed357 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:10.057217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d30375-e3ce-4f88-99d3-7f2086815185 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:10.057217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d30375-e3ce-4f88-99d3-7f2086815185 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:10.057217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d30375-e3ce-4f88-99d3-7f2086815185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:10.093006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe795f1-4b89-4dec-b8d4-6397716ed357 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:10.097810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20a074-8402-45ac-bf72-54e5468b540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:10.098529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20a074-8402-45ac-bf72-54e5468b540f admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:10.098862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20a074-8402-45ac-bf72-54e5468b540f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:10.099140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20a074-8402-45ac-bf72-54e5468b540f admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:10.099851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20a074-8402-45ac-bf72-54e5468b540f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:10.106077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20a074-8402-45ac-bf72-54e5468b540f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:10.123993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-71d30375-e3ce-4f88-99d3-7f2086815185 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:10.125827 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-71d30375-e3ce-4f88-99d3-7f2086815185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:10.152897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 860/1709] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:10.161081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8749b56-126a-4978-b660-50f62b6adaf8 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:10.166796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8749b56-126a-4978-b660-50f62b6adaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:10.169891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8749b56-126a-4978-b660-50f62b6adaf8 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:10.171634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 861/1710] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jul 14 11:44:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:44:10.175809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8749b56-126a-4978-b660-50f62b6adaf8 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:10.211506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 850/1711] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:10 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:10.212360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20a074-8402-45ac-bf72-54e5468b540f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:10.220431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8749b56-126a-4978-b660-50f62b6adaf8 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:10.282877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:10.366015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20a074-8402-45ac-bf72-54e5468b540f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:10.384364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b36cd1-250b-4d21-a7c7-94e6fc1202e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:10.385111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b36cd1-250b-4d21-a7c7-94e6fc1202e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:10.385422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b36cd1-250b-4d21-a7c7-94e6fc1202e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:10.385692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b36cd1-250b-4d21-a7c7-94e6fc1202e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:10.386527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b36cd1-250b-4d21-a7c7-94e6fc1202e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:10.386837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b36cd1-250b-4d21-a7c7-94e6fc1202e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:10.392415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b36cd1-250b-4d21-a7c7-94e6fc1202e0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:10.430562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 851/1712] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:10.432036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9926-6889-4f19-912a-818b9a877f7a None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:10.432321 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9926-6889-4f19-912a-818b9a877f7a None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:10.432594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9926-6889-4f19-912a-818b9a877f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:10.461134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:10.465014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a430d-3af9-45e8-9404-87a0f6146e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:10.465734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a430d-3af9-45e8-9404-87a0f6146e26 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:10.466013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a430d-3af9-45e8-9404-87a0f6146e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:10.467131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a430d-3af9-45e8-9404-87a0f6146e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:10.471044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a430d-3af9-45e8-9404-87a0f6146e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:10.475283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a430d-3af9-45e8-9404-87a0f6146e26 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:10.481116 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-a0fa9926-6889-4f19-912a-818b9a877f7a None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:10.495766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a0fa9926-6889-4f19-912a-818b9a877f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:10.511638 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 862/1713] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:10.511638 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a430d-3af9-45e8-9404-87a0f6146e26 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:10.538836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a430d-3af9-45e8-9404-87a0f6146e26 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:10.550571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1c941-8331-4c2e-b525-3eb07a148a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:10.551252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1c941-8331-4c2e-b525-3eb07a148a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:10.551532 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1c941-8331-4c2e-b525-3eb07a148a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:10.551802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1c941-8331-4c2e-b525-3eb07a148a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:10.552499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1c941-8331-4c2e-b525-3eb07a148a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:10.557616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1c941-8331-4c2e-b525-3eb07a148a04 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:10.642201 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 852/1714] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:44:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:10.642438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa9926-6889-4f19-912a-818b9a877f7a None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:10.659708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa9926-6889-4f19-912a-818b9a877f7a None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:10.665147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e488-52dd-430b-bb54-1ec26c4fb71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:10.670650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e488-52dd-430b-bb54-1ec26c4fb71f admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:10.670650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e488-52dd-430b-bb54-1ec26c4fb71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:10.670650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e488-52dd-430b-bb54-1ec26c4fb71f admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:10.671393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086e488-52dd-430b-bb54-1ec26c4fb71f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:10.674515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086e488-52dd-430b-bb54-1ec26c4fb71f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:10.701126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 863/1715] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:10 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:10.702439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1c941-8331-4c2e-b525-3eb07a148a04 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:10.720687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1c941-8331-4c2e-b525-3eb07a148a04 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:10.728091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e278913-db4b-4767-b24c-e91fb8e0f818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:10.728091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278913-db4b-4767-b24c-e91fb8e0f818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:10.728091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278913-db4b-4767-b24c-e91fb8e0f818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:10.728091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278913-db4b-4767-b24c-e91fb8e0f818 admin admin] PATH_INFO: `/v3/projects/7ec2443e342f45bbb5140cc2168dab1e/users/69d2b241046f4c7fbedf5c765d92f16d/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:10.750079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e278913-db4b-4767-b24c-e91fb8e0f818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2443e342f45bbb5140cc2168dab1e, user_id=69d2b241046f4c7fbedf5c765d92f16d, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:10.753188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e278913-db4b-4767-b24c-e91fb8e0f818 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:10.873093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 853/1716] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:10 2022] POST /identity/v3/projects => generated 367 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:10.873093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e488-52dd-430b-bb54-1ec26c4fb71f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:10.873093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c086e488-52dd-430b-bb54-1ec26c4fb71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:10.986626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 864/1717] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:10 2022] PUT /identity/v3/projects/7ec2443e342f45bbb5140cc2168dab1e/users/69d2b241046f4c7fbedf5c765d92f16d/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:10.988108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e278913-db4b-4767-b24c-e91fb8e0f818 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:11.107992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e488-52dd-430b-bb54-1ec26c4fb71f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:11.160489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e278913-db4b-4767-b24c-e91fb8e0f818 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:11.166526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006972fe-c085-45ad-9be9-e24a54a0e7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:11.167478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006972fe-c085-45ad-9be9-e24a54a0e7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:11.168355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006972fe-c085-45ad-9be9-e24a54a0e7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:11.168464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006972fe-c085-45ad-9be9-e24a54a0e7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:11.170146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006972fe-c085-45ad-9be9-e24a54a0e7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:11.176309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006972fe-c085-45ad-9be9-e24a54a0e7a4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:11.194592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 865/1718] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:11.194592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006972fe-c085-45ad-9be9-e24a54a0e7a4 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:11.215395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006972fe-c085-45ad-9be9-e24a54a0e7a4 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:11.223299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d0831-50f2-4bdc-946e-e9fb047d8504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:11.226337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d0831-50f2-4bdc-946e-e9fb047d8504 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:11.228067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d0831-50f2-4bdc-946e-e9fb047d8504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:11.229279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d0831-50f2-4bdc-946e-e9fb047d8504 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:11.231632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d0831-50f2-4bdc-946e-e9fb047d8504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:11.235895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d0831-50f2-4bdc-946e-e9fb047d8504 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:11.255217 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e666a2-d4c7-4e04-8af2-22a84b09e8d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1a7213cbf6448b7ba6570a52a4c7c0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:11.257173 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e666a2-d4c7-4e04-8af2-22a84b09e8d5 service glance] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:11.257955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e666a2-d4c7-4e04-8af2-22a84b09e8d5 service glance] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:11.259031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e666a2-d4c7-4e04-8af2-22a84b09e8d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:11.402206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e666a2-d4c7-4e04-8af2-22a84b09e8d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:11.403842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e666a2-d4c7-4e04-8af2-22a84b09e8d5 service glance] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:11.417880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 854/1719] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:11.419976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9c9f-c513-42e7-894a-9a38567678bb None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:11.420417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9c9f-c513-42e7-894a-9a38567678bb None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:11.420665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9c9f-c513-42e7-894a-9a38567678bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:11.455504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-79cf9c9f-c513-42e7-894a-9a38567678bb None None] MFA Rules not processed for user `69d2b241046f4c7fbedf5c765d92f16d`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:11.456858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-79cf9c9f-c513-42e7-894a-9a38567678bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:11.490192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 866/1720] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:11 2022] POST /identity/v3/users => generated 284 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:11.491245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d0831-50f2-4bdc-946e-e9fb047d8504 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:11.527763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d0831-50f2-4bdc-946e-e9fb047d8504 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:11.539851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310df42-91c7-4f6e-af28-7bd323e67c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:11.541361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310df42-91c7-4f6e-af28-7bd323e67c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:11.545332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310df42-91c7-4f6e-af28-7bd323e67c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:11.545332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310df42-91c7-4f6e-af28-7bd323e67c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:11.545332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310df42-91c7-4f6e-af28-7bd323e67c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:11.546843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310df42-91c7-4f6e-af28-7bd323e67c94 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:11.565002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 867/1721] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:11.566234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310df42-91c7-4f6e-af28-7bd323e67c94 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:11.586834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310df42-91c7-4f6e-af28-7bd323e67c94 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:11.613287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fe408-a94e-4b5f-b7f5-18e9e56087ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:11.616434 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fe408-a94e-4b5f-b7f5-18e9e56087ec admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:11.616434 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fe408-a94e-4b5f-b7f5-18e9e56087ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:11.618449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fe408-a94e-4b5f-b7f5-18e9e56087ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:11.620414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fe408-a94e-4b5f-b7f5-18e9e56087ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:11.623477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:11.626552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fe408-a94e-4b5f-b7f5-18e9e56087ec admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:11.646770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 868/1722] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:11.648669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fe408-a94e-4b5f-b7f5-18e9e56087ec admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:11.670478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fe408-a94e-4b5f-b7f5-18e9e56087ec admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:11.685631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad2582-0f6a-48a9-bd7c-973b15e4da76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:11.686802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad2582-0f6a-48a9-bd7c-973b15e4da76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:11.687197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad2582-0f6a-48a9-bd7c-973b15e4da76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:11.687662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad2582-0f6a-48a9-bd7c-973b15e4da76 admin admin] PATH_INFO: `/v3/projects/27b77debca034122a25fff253dc95518/users/1f3e893029a14a08b9fd83ec6c7eed86/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:11.698433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 855/1723] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:11 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:11.702318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf9c9f-c513-42e7-894a-9a38567678bb None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:11.727930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad2582-0f6a-48a9-bd7c-973b15e4da76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b77debca034122a25fff253dc95518, user_id=1f3e893029a14a08b9fd83ec6c7eed86, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:11.730386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf9c9f-c513-42e7-894a-9a38567678bb None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:11.731636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad2582-0f6a-48a9-bd7c-973b15e4da76 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:11.733708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:11.751773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5d2fa-a97d-4d23-b128-5c75347d03df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:11.752161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5d2fa-a97d-4d23-b128-5c75347d03df admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:11.752448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5d2fa-a97d-4d23-b128-5c75347d03df admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:11.752721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5d2fa-a97d-4d23-b128-5c75347d03df admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:11.753502 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5d2fa-a97d-4d23-b128-5c75347d03df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:11.757437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5d2fa-a97d-4d23-b128-5c75347d03df admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:11.784723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 856/1724] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:11.788142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5d2fa-a97d-4d23-b128-5c75347d03df admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:11.809426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5d2fa-a97d-4d23-b128-5c75347d03df admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:11.823594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 869/1725] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:11 2022] PUT /identity/v3/projects/27b77debca034122a25fff253dc95518/users/1f3e893029a14a08b9fd83ec6c7eed86/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:11.824430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad2582-0f6a-48a9-bd7c-973b15e4da76 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:11.824559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db123e-ca61-45d2-8b1e-031859a15786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:11.825844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db123e-ca61-45d2-8b1e-031859a15786 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:11.828735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db123e-ca61-45d2-8b1e-031859a15786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:11.828735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db123e-ca61-45d2-8b1e-031859a15786 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:11.828735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db123e-ca61-45d2-8b1e-031859a15786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:11.831700 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db123e-ca61-45d2-8b1e-031859a15786 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:11.951593 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 857/1726] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:11 2022] POST /identity/v3/projects => generated 357 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:11.953343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e614-9784-49e4-a2e8-754ed5f19c2c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:11.953638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e614-9784-49e4-a2e8-754ed5f19c2c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:11.953919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e614-9784-49e4-a2e8-754ed5f19c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:12.008546 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad2582-0f6a-48a9-bd7c-973b15e4da76 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:12.016860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9986e614-9784-49e4-a2e8-754ed5f19c2c None None] MFA Rules not processed for user `1f3e893029a14a08b9fd83ec6c7eed86`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:12.019901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9986e614-9784-49e4-a2e8-754ed5f19c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:12.147570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbaa9c-b31b-4ccd-afd8-92d65fb2b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:12.149441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbaa9c-b31b-4ccd-afd8-92d65fb2b6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:12.149441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbaa9c-b31b-4ccd-afd8-92d65fb2b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:12.149767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbaa9c-b31b-4ccd-afd8-92d65fb2b6d5 admin admin] PATH_INFO: `/v3/projects/ef766347aba243a498aeae29dce0a68f/users/27be9eeb8e6c4065b60c694f58484def/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:12.168318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 858/1727] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:12.168318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e614-9784-49e4-a2e8-754ed5f19c2c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:12.180868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e614-9784-49e4-a2e8-754ed5f19c2c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:12.190269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2f98d-790c-4f25-96c7-7d5acc4292e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:12.193360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2f98d-790c-4f25-96c7-7d5acc4292e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:12.195095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2f98d-790c-4f25-96c7-7d5acc4292e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:12.195095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2f98d-790c-4f25-96c7-7d5acc4292e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:12.197866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2f98d-790c-4f25-96c7-7d5acc4292e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:12.206463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2f98d-790c-4f25-96c7-7d5acc4292e3 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:12.212070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbaa9c-b31b-4ccd-afd8-92d65fb2b6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef766347aba243a498aeae29dce0a68f, user_id=27be9eeb8e6c4065b60c694f58484def, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:12.220180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbaa9c-b31b-4ccd-afd8-92d65fb2b6d5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:12.238620 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:12.294646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 859/1728] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:12 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:12.297835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c1fac-a99e-4725-8e31-8908b17db414 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:12.298206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c1fac-a99e-4725-8e31-8908b17db414 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:12.298560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c1fac-a99e-4725-8e31-8908b17db414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:12.323167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 870/1729] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:11 2022] PUT /identity/v3/projects/ef766347aba243a498aeae29dce0a68f/users/27be9eeb8e6c4065b60c694f58484def/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:12.323167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbaa9c-b31b-4ccd-afd8-92d65fb2b6d5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:12.454561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-1a4c1fac-a99e-4725-8e31-8908b17db414 None None] MFA Rules not processed for user `899cbac79d874b308b500651bb40b2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:12.457339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1a4c1fac-a99e-4725-8e31-8908b17db414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:12.519240 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbaa9c-b31b-4ccd-afd8-92d65fb2b6d5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:12.530579 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca36cba-8abc-4fd2-8a36-fa59e4427e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:12.533389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca36cba-8abc-4fd2-8a36-fa59e4427e18 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:12.533773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca36cba-8abc-4fd2-8a36-fa59e4427e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:12.534032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca36cba-8abc-4fd2-8a36-fa59e4427e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:12.591624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:12.630545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 860/1730] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:12.638010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c1fac-a99e-4725-8e31-8908b17db414 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:12.708692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca36cba-8abc-4fd2-8a36-fa59e4427e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:12.710534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca36cba-8abc-4fd2-8a36-fa59e4427e18 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:12.733546 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 871/1731] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:12.740950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b807b-2535-4ee6-87e5-d73e4b173592 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:12.740950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b807b-2535-4ee6-87e5-d73e4b173592 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:12.740950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b807b-2535-4ee6-87e5-d73e4b173592 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:12.741729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 872/1732] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jul 14 11:44:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:44:12.743245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a76a8-c0e4-4cd4-bbaa-ce6ba685215a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:12.743669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a76a8-c0e4-4cd4-bbaa-ce6ba685215a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:12.744894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a76a8-c0e4-4cd4-bbaa-ce6ba685215a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:12.823676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-735a76a8-c0e4-4cd4-bbaa-ce6ba685215a None None] MFA Rules not processed for user `27be9eeb8e6c4065b60c694f58484def`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:12.824836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-735a76a8-c0e4-4cd4-bbaa-ce6ba685215a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:12.831172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c1fac-a99e-4725-8e31-8908b17db414 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:12.845662 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eab0f2-f944-4e0d-b69d-0d8d98a904f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:12.859211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eab0f2-f944-4e0d-b69d-0d8d98a904f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:12.859843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eab0f2-f944-4e0d-b69d-0d8d98a904f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:12.869858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eab0f2-f944-4e0d-b69d-0d8d98a904f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:12.872700 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eab0f2-f944-4e0d-b69d-0d8d98a904f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:12.874628 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eab0f2-f944-4e0d-b69d-0d8d98a904f9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:12.913104 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 861/1733] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:12.924865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eab0f2-f944-4e0d-b69d-0d8d98a904f9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:13.094618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 873/1734] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:13.096095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214b4c2-2e3d-49b2-b1d1-50939a23df0c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:13.096384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214b4c2-2e3d-49b2-b1d1-50939a23df0c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:13.096656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214b4c2-2e3d-49b2-b1d1-50939a23df0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:13.125808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eab0f2-f944-4e0d-b69d-0d8d98a904f9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:13.139235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f69d0-072e-4c17-a526-61cd38344ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:13.141263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f69d0-072e-4c17-a526-61cd38344ce6 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:13.141549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f69d0-072e-4c17-a526-61cd38344ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:13.141825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f69d0-072e-4c17-a526-61cd38344ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:13.155204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f69d0-072e-4c17-a526-61cd38344ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:13.158440 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f69d0-072e-4c17-a526-61cd38344ce6 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:13.176830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9214b4c2-2e3d-49b2-b1d1-50939a23df0c None None] MFA Rules not processed for user `481ef56b1cfa465a8efc59736a34e412`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:13.178095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9214b4c2-2e3d-49b2-b1d1-50939a23df0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:13.186220 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 862/1735] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:13.187735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f69d0-072e-4c17-a526-61cd38344ce6 service nova] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:13.196951 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:13.211729 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f69d0-072e-4c17-a526-61cd38344ce6 service nova] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:13.222075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:13.223118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:13.223238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:13.223518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] PATH_INFO: `/v3/users/1f3e893029a14a08b9fd83ec6c7eed86` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:13.225983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3e893029a14a08b9fd83ec6c7eed86)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:13.231438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:13.274132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 874/1736] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:44:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:13.275997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214b4c2-2e3d-49b2-b1d1-50939a23df0c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:13.299920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214b4c2-2e3d-49b2-b1d1-50939a23df0c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:13.308908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2f7ce-8ebd-4ac3-806b-4d3a3913f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:13.311822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2f7ce-8ebd-4ac3-806b-4d3a3913f734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:13.312343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2f7ce-8ebd-4ac3-806b-4d3a3913f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:13.312776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2f7ce-8ebd-4ac3-806b-4d3a3913f734 admin admin] PATH_INFO: `/v3/projects/2f87c9c6faef479fa7febf9d21c7fa0f/users/8d2e4db18d974872816b84cc616a2de8/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:13.338367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2f7ce-8ebd-4ac3-806b-4d3a3913f734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f87c9c6faef479fa7febf9d21c7fa0f, user_id=8d2e4db18d974872816b84cc616a2de8, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:13.342036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2f7ce-8ebd-4ac3-806b-4d3a3913f734 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:13.458073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3e893029a14a08b9fd83ec6c7eed86'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:13.459865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3e893029a14a08b9fd83ec6c7eed86'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:13.479892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3e893029a14a08b9fd83ec6c7eed86'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:13.484723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 875/1737] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:13 2022] PUT /identity/v3/projects/2f87c9c6faef479fa7febf9d21c7fa0f/users/8d2e4db18d974872816b84cc616a2de8/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:13.486449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2f7ce-8ebd-4ac3-806b-4d3a3913f734 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:13.490242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bcb2f7ce-8ebd-4ac3-806b-4d3a3913f734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:13.492897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3e893029a14a08b9fd83ec6c7eed86'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:13.562837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3e893029a14a08b9fd83ec6c7eed86'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:13.585524 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 863/1738] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:13 2022] DELETE /identity/v3/users/1f3e893029a14a08b9fd83ec6c7eed86 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:13.592618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2f7ce-8ebd-4ac3-806b-4d3a3913f734 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:13.594824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:13.612043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:13.716913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:13.751055 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ef308-159f-4f68-886d-0a81bc07d3ee admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:13.755184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2faa09-b80d-41ac-a127-04ef61a467e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:13.755596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbae017-ef8b-49f9-a9ec-9a5e6f97d9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:13.755853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2faa09-b80d-41ac-a127-04ef61a467e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:13.756136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2faa09-b80d-41ac-a127-04ef61a467e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:13.756303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbae017-ef8b-49f9-a9ec-9a5e6f97d9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:13.756401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2faa09-b80d-41ac-a127-04ef61a467e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:13.756554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbae017-ef8b-49f9-a9ec-9a5e6f97d9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:13.756836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbae017-ef8b-49f9-a9ec-9a5e6f97d9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:13.759226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6b2faa09-b80d-41ac-a127-04ef61a467e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:13.760166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbae017-ef8b-49f9-a9ec-9a5e6f97d9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:13.764992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbae017-ef8b-49f9-a9ec-9a5e6f97d9c0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:13.799430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2faa09-b80d-41ac-a127-04ef61a467e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:13.800991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2faa09-b80d-41ac-a127-04ef61a467e0 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:13.834545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 864/1739] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:13.836161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82225b9a-0137-4755-ad74-6369c5fff97c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:13.836559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82225b9a-0137-4755-ad74-6369c5fff97c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:13.836947 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82225b9a-0137-4755-ad74-6369c5fff97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:13.912414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 876/1740] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:13 2022] POST /identity/v3/projects => generated 369 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:13.912414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbae017-ef8b-49f9-a9ec-9a5e6f97d9c0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:13.921206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bdbae017-ef8b-49f9-a9ec-9a5e6f97d9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:13.922226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-82225b9a-0137-4755-ad74-6369c5fff97c None None] MFA Rules not processed for user `8773c88f88614a9d903f667305a217e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:13.923437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-82225b9a-0137-4755-ad74-6369c5fff97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:13.966348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:14.073954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 865/1741] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:14.075753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406979a-a499-4fad-b1de-195328e23050 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:14.078830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406979a-a499-4fad-b1de-195328e23050 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:14.079386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406979a-a499-4fad-b1de-195328e23050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:14.155785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbae017-ef8b-49f9-a9ec-9a5e6f97d9c0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:14.165781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-5406979a-a499-4fad-b1de-195328e23050 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:14.166852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0846dcb-ea3b-48b8-986a-d1673a164bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:14.167682 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0846dcb-ea3b-48b8-986a-d1673a164bed admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:14.173071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0846dcb-ea3b-48b8-986a-d1673a164bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:14.173461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5406979a-a499-4fad-b1de-195328e23050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:14.173553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0846dcb-ea3b-48b8-986a-d1673a164bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:14.173553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0846dcb-ea3b-48b8-986a-d1673a164bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:14.173553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0846dcb-ea3b-48b8-986a-d1673a164bed admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:14.194807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 866/1742] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:14.195718 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406979a-a499-4fad-b1de-195328e23050 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:14.208393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 877/1743] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:14.209542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0846dcb-ea3b-48b8-986a-d1673a164bed admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:14.217348 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f0846dcb-ea3b-48b8-986a-d1673a164bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:14.231606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406979a-a499-4fad-b1de-195328e23050 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:14.245505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f936bb-ef01-434f-b155-2bcf6787e874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:14.246207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f936bb-ef01-434f-b155-2bcf6787e874 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:14.246494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f936bb-ef01-434f-b155-2bcf6787e874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:14.246963 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f936bb-ef01-434f-b155-2bcf6787e874 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:14.248429 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f936bb-ef01-434f-b155-2bcf6787e874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:14.250499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f936bb-ef01-434f-b155-2bcf6787e874 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:14.410888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 867/1744] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:14 2022] POST /identity/v3/users => generated 291 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:14.411732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f936bb-ef01-434f-b155-2bcf6787e874 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:14.412476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0846dcb-ea3b-48b8-986a-d1673a164bed admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:14.423379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fd7f9-6b83-4ff7-8615-42259d62a170 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:14.424059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fd7f9-6b83-4ff7-8615-42259d62a170 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:14.424339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fd7f9-6b83-4ff7-8615-42259d62a170 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:14.424612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fd7f9-6b83-4ff7-8615-42259d62a170 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:14.448578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f936bb-ef01-434f-b155-2bcf6787e874 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:14.460567 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fd7f9-6b83-4ff7-8615-42259d62a170 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:14.462229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fd7f9-6b83-4ff7-8615-42259d62a170 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:14.467081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f2608-0157-4689-a942-69ed358b5739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:14.468027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f2608-0157-4689-a942-69ed358b5739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:14.468313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f2608-0157-4689-a942-69ed358b5739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:14.468616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f2608-0157-4689-a942-69ed358b5739 admin admin] PATH_INFO: `/v3/projects/2f87c9c6faef479fa7febf9d21c7fa0f/users/8d2e4db18d974872816b84cc616a2de8/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:14.474661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f2608-0157-4689-a942-69ed358b5739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f87c9c6faef479fa7febf9d21c7fa0f, user_id=8d2e4db18d974872816b84cc616a2de8, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:14.477929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f2608-0157-4689-a942-69ed358b5739 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:14.494613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 878/1745] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:14.504704 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fd7f9-6b83-4ff7-8615-42259d62a170 service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:14.516848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 868/1746] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:14 2022] PUT /identity/v3/projects/2f87c9c6faef479fa7febf9d21c7fa0f/users/8d2e4db18d974872816b84cc616a2de8/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:14.522044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f2608-0157-4689-a942-69ed358b5739 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:14.672172 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fd7f9-6b83-4ff7-8615-42259d62a170 service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:14.678029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:14.705325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f2608-0157-4689-a942-69ed358b5739 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:14.721678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b83e27-0bc5-4911-b835-f9edad4c8d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:14.722531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83e27-0bc5-4911-b835-f9edad4c8d2d admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:14.722812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83e27-0bc5-4911-b835-f9edad4c8d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:14.723086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83e27-0bc5-4911-b835-f9edad4c8d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:14.723962 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b83e27-0bc5-4911-b835-f9edad4c8d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:14.725619 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b83e27-0bc5-4911-b835-f9edad4c8d2d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:14.753231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 869/1747] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:14.755491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f2ed3-3974-4a7f-9a4b-af31e27e1543 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:14.755611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f2ed3-3974-4a7f-9a4b-af31e27e1543 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:14.757508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f2ed3-3974-4a7f-9a4b-af31e27e1543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:14.788351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423c96c-8503-4a6a-a604-623ce54d4dfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:14.791645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423c96c-8503-4a6a-a604-623ce54d4dfa service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:14.791934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423c96c-8503-4a6a-a604-623ce54d4dfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:14.792207 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423c96c-8503-4a6a-a604-623ce54d4dfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:14.826365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423c96c-8503-4a6a-a604-623ce54d4dfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:14.832073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423c96c-8503-4a6a-a604-623ce54d4dfa service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:14.837932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-104f2ed3-3974-4a7f-9a4b-af31e27e1543 None None] MFA Rules not processed for user `8d2e4db18d974872816b84cc616a2de8`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:14.837932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-104f2ed3-3974-4a7f-9a4b-af31e27e1543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:14.858130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 879/1748] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:14.860161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423c96c-8503-4a6a-a604-623ce54d4dfa service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:14.894008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423c96c-8503-4a6a-a604-623ce54d4dfa service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:14.909560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd3f91-f441-4f13-8741-39cd2c43baa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:14.910809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd3f91-f441-4f13-8741-39cd2c43baa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:14.911236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd3f91-f441-4f13-8741-39cd2c43baa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:14.912752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd3f91-f441-4f13-8741-39cd2c43baa6 admin admin] PATH_INFO: `/v3/projects/8066f7e6b407475080059679d0c0b488/users/92d19188fbf14ded98794aa92ad6f560/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:14.970354 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd3f91-f441-4f13-8741-39cd2c43baa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8066f7e6b407475080059679d0c0b488, user_id=92d19188fbf14ded98794aa92ad6f560, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:14.987264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd3f91-f441-4f13-8741-39cd2c43baa6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:15.074179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 880/1749] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:14 2022] PUT /identity/v3/projects/8066f7e6b407475080059679d0c0b488/users/92d19188fbf14ded98794aa92ad6f560/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:15.076288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fc683-d9e3-4bb7-a94d-8ad2054d9958 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:15.076288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fc683-d9e3-4bb7-a94d-8ad2054d9958 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:15.076288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fc683-d9e3-4bb7-a94d-8ad2054d9958 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:15.077459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 881/1750] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jul 14 11:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:44:15.089466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb72a6a-eae1-4dfe-b328-f2eaa10a3bd7 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:15.092760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb72a6a-eae1-4dfe-b328-f2eaa10a3bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:15.092869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb72a6a-eae1-4dfe-b328-f2eaa10a3bd7 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:15.101436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 870/1751] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:14 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:15.105149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 882/1752] 10.222.0.40 () {54 vars in 960 bytes} [Thu Jul 14 11:44:15 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:44:15.105321 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f2ed3-3974-4a7f-9a4b-af31e27e1543 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:15.106434 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8f72a-8ace-4f82-b613-9731c66fdd73 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:15.106434 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8f72a-8ace-4f82-b613-9731c66fdd73 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:15.106628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8f72a-8ace-4f82-b613-9731c66fdd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:15.154307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f2ed3-3974-4a7f-9a4b-af31e27e1543 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:15.159252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:15.199410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f6d8f72a-8ace-4f82-b613-9731c66fdd73 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:15.199410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f6d8f72a-8ace-4f82-b613-9731c66fdd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:15.403995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210ed44-01dd-41fb-aefe-c5fe8823e305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:15.404967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210ed44-01dd-41fb-aefe-c5fe8823e305 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:15.405678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210ed44-01dd-41fb-aefe-c5fe8823e305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:15.406109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210ed44-01dd-41fb-aefe-c5fe8823e305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:15.407028 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210ed44-01dd-41fb-aefe-c5fe8823e305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:15.408907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210ed44-01dd-41fb-aefe-c5fe8823e305 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:15.449909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 883/1753] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:44:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:15.451260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbc5b0-c147-415a-9cd1-5fe1d068c32b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:15.451544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbc5b0-c147-415a-9cd1-5fe1d068c32b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:15.451821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbc5b0-c147-415a-9cd1-5fe1d068c32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:15.486382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 871/1754] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:15.486382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210ed44-01dd-41fb-aefe-c5fe8823e305 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:15.486382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7210ed44-01dd-41fb-aefe-c5fe8823e305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:15.517932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d9bbc5b0-c147-415a-9cd1-5fe1d068c32b None None] MFA Rules not processed for user `fa879d581c6d44e18a8703e66ffbd549`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:15.521313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d9bbc5b0-c147-415a-9cd1-5fe1d068c32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:15.630342 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210ed44-01dd-41fb-aefe-c5fe8823e305 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:15.640870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db040be2-bcde-4e95-a7f9-b7a1dddf50c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:15.640870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040be2-bcde-4e95-a7f9-b7a1dddf50c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:15.640870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040be2-bcde-4e95-a7f9-b7a1dddf50c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:15.640870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040be2-bcde-4e95-a7f9-b7a1dddf50c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:15.750654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 884/1755] 10.222.0.40 () {58 vars in 1057 bytes} [Thu Jul 14 11:44:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:15.755878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbc5b0-c147-415a-9cd1-5fe1d068c32b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:15.768700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbc5b0-c147-415a-9cd1-5fe1d068c32b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:15.779596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe804ca-d618-41a5-8f2a-6d085d7fb300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:15.784396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe804ca-d618-41a5-8f2a-6d085d7fb300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:15.784396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe804ca-d618-41a5-8f2a-6d085d7fb300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:15.784396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe804ca-d618-41a5-8f2a-6d085d7fb300 admin admin] PATH_INFO: `/v3/projects/8066f7e6b407475080059679d0c0b488/users/92d19188fbf14ded98794aa92ad6f560/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:15.785209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe804ca-d618-41a5-8f2a-6d085d7fb300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8066f7e6b407475080059679d0c0b488, user_id=92d19188fbf14ded98794aa92ad6f560, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:15.788203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe804ca-d618-41a5-8f2a-6d085d7fb300 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:15.814492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db040be2-bcde-4e95-a7f9-b7a1dddf50c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:15.816223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db040be2-bcde-4e95-a7f9-b7a1dddf50c7 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:15.816485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 885/1756] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:15 2022] PUT /identity/v3/projects/8066f7e6b407475080059679d0c0b488/users/92d19188fbf14ded98794aa92ad6f560/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:15.817571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe804ca-d618-41a5-8f2a-6d085d7fb300 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:15.827939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bfe804ca-d618-41a5-8f2a-6d085d7fb300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:15.842309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 872/1757] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:15 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:15.842309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db040be2-bcde-4e95-a7f9-b7a1dddf50c7 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:15.867344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db040be2-bcde-4e95-a7f9-b7a1dddf50c7 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:16.008185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe804ca-d618-41a5-8f2a-6d085d7fb300 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:16.016079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dced9e-3a30-4866-9a8f-0ef362d2f541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:16.017011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dced9e-3a30-4866-9a8f-0ef362d2f541 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:16.018335 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dced9e-3a30-4866-9a8f-0ef362d2f541 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:16.018820 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dced9e-3a30-4866-9a8f-0ef362d2f541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:16.023768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded15b6-5b3f-4d29-b1aa-37d7a9bf7aaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:16.024755 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded15b6-5b3f-4d29-b1aa-37d7a9bf7aaa service placement] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:16.025295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded15b6-5b3f-4d29-b1aa-37d7a9bf7aaa service placement] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:16.026396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded15b6-5b3f-4d29-b1aa-37d7a9bf7aaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:16.030757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0ded15b6-5b3f-4d29-b1aa-37d7a9bf7aaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:16.059613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded15b6-5b3f-4d29-b1aa-37d7a9bf7aaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:16.062176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded15b6-5b3f-4d29-b1aa-37d7a9bf7aaa service placement] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:16.078393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 873/1758] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:16.080152 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe7dc-b6bb-4f89-b5ee-a08e564db6eb None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:16.080645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe7dc-b6bb-4f89-b5ee-a08e564db6eb None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:16.081184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe7dc-b6bb-4f89-b5ee-a08e564db6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:16.114354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-6fdbe7dc-b6bb-4f89-b5ee-a08e564db6eb None None] MFA Rules not processed for user `92d19188fbf14ded98794aa92ad6f560`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:16.115830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6fdbe7dc-b6bb-4f89-b5ee-a08e564db6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:16.214880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dced9e-3a30-4866-9a8f-0ef362d2f541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:16.218784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dced9e-3a30-4866-9a8f-0ef362d2f541 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:16.220326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 874/1759] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:16 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:16.227133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe7dc-b6bb-4f89-b5ee-a08e564db6eb None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:16.241068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 886/1760] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:16.242058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dced9e-3a30-4866-9a8f-0ef362d2f541 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:16.243666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-32dced9e-3a30-4866-9a8f-0ef362d2f541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:16.347650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:16.389621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe7dc-b6bb-4f89-b5ee-a08e564db6eb None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:16.397474 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eff2a9-1595-4248-a786-30681e022334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:16.398153 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eff2a9-1595-4248-a786-30681e022334 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:16.399658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eff2a9-1595-4248-a786-30681e022334 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:16.399945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eff2a9-1595-4248-a786-30681e022334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:16.404000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f8eff2a9-1595-4248-a786-30681e022334 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:16.412213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dced9e-3a30-4866-9a8f-0ef362d2f541 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:16.428586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:16.429978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:16.430266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:16.430543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] PATH_INFO: `/v3/projects/27b77debca034122a25fff253dc95518` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:16.432473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b77debca034122a25fff253dc95518)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:16.435060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:16.570773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eff2a9-1595-4248-a786-30681e022334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:16.573605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eff2a9-1595-4248-a786-30681e022334 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:16.612108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 875/1761] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:16 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:16.613416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eff2a9-1595-4248-a786-30681e022334 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:16.618229 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f8eff2a9-1595-4248-a786-30681e022334 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:16.624197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b77debca034122a25fff253dc95518', 'request_id': 'req-5ae6264a-bb41-462c-b690-f46d6f137e22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:16.642096 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eff2a9-1595-4248-a786-30681e022334 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:16.727946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b77debca034122a25fff253dc95518', 'request_id': 'req-5ae6264a-bb41-462c-b690-f46d6f137e22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:16.741276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b77debca034122a25fff253dc95518', 'request_id': 'req-5ae6264a-bb41-462c-b690-f46d6f137e22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:16.747871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3e3f5-2494-4aa7-91e7-afa394f0dff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:16.748651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3e3f5-2494-4aa7-91e7-afa394f0dff6 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:16.749106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3e3f5-2494-4aa7-91e7-afa394f0dff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:16.750201 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3e3f5-2494-4aa7-91e7-afa394f0dff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:16.759071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] The token cache is being invalidate because project 27b77debca034122a25fff253dc95518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:16.759914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:16.761066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5ae6264a-bb41-462c-b690-f46d6f137e22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:16.764881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 887/1762] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:16 2022] DELETE /identity/v3/projects/27b77debca034122a25fff253dc95518 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:16.769590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7f911-9ecb-452c-bb1f-581fe5b74a20 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:16.769881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7f911-9ecb-452c-bb1f-581fe5b74a20 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:16.770161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7f911-9ecb-452c-bb1f-581fe5b74a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:16.803543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-bdd7f911-9ecb-452c-bb1f-581fe5b74a20 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:16.808066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bdd7f911-9ecb-452c-bb1f-581fe5b74a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:16.852856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:16.896326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3e3f5-2494-4aa7-91e7-afa394f0dff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:16.896326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3e3f5-2494-4aa7-91e7-afa394f0dff6 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:16.908907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 876/1763] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:16 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:16.910387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bdb2f-1129-4086-82eb-c17f9ce04355 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:16.911320 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bdb2f-1129-4086-82eb-c17f9ce04355 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:16.911320 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bdb2f-1129-4086-82eb-c17f9ce04355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:16.937117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c60bdb2f-1129-4086-82eb-c17f9ce04355 None None] MFA Rules not processed for user `246f07024efa444da74f2b898f426b46`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:16.939407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c60bdb2f-1129-4086-82eb-c17f9ce04355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:16.979245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 888/1764] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:16.985263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098bbce-7a8d-4ba2-9425-05f3d1029e81 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:16.985613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098bbce-7a8d-4ba2-9425-05f3d1029e81 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:16.985898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098bbce-7a8d-4ba2-9425-05f3d1029e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:17.015984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6098bbce-7a8d-4ba2-9425-05f3d1029e81 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:17.017672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6098bbce-7a8d-4ba2-9425-05f3d1029e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:17.034064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 889/1765] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:17.038411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098bbce-7a8d-4ba2-9425-05f3d1029e81 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:17.064356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098bbce-7a8d-4ba2-9425-05f3d1029e81 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:17.081246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05bb9c-e248-4ce9-bd80-9b394bff8f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:17.081940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05bb9c-e248-4ce9-bd80-9b394bff8f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:17.082229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05bb9c-e248-4ce9-bd80-9b394bff8f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:17.082506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05bb9c-e248-4ce9-bd80-9b394bff8f52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:17.083254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05bb9c-e248-4ce9-bd80-9b394bff8f52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:17.083568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05bb9c-e248-4ce9-bd80-9b394bff8f52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:17.086898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05bb9c-e248-4ce9-bd80-9b394bff8f52 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:17.092248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 877/1766] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:17.112947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 890/1767] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:17.114112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bdb2f-1129-4086-82eb-c17f9ce04355 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:17.125999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bdb2f-1129-4086-82eb-c17f9ce04355 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:17.134307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8426f-36f2-43dc-aad1-744466d0419d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:17.135778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8426f-36f2-43dc-aad1-744466d0419d admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:17.139806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8426f-36f2-43dc-aad1-744466d0419d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:17.139806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8426f-36f2-43dc-aad1-744466d0419d admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:17.139806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8426f-36f2-43dc-aad1-744466d0419d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:17.140825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8426f-36f2-43dc-aad1-744466d0419d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:17.250537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:17.277071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05bb9c-e248-4ce9-bd80-9b394bff8f52 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:17.278543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 878/1768] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:17 2022] POST /identity/v3/projects => generated 369 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:17.314660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05bb9c-e248-4ce9-bd80-9b394bff8f52 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:17.464452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995fad0-4863-4807-9bc1-41bdf4fadb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:17.465796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995fad0-4863-4807-9bc1-41bdf4fadb25 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:17.471161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995fad0-4863-4807-9bc1-41bdf4fadb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:17.471161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995fad0-4863-4807-9bc1-41bdf4fadb25 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:17.471161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995fad0-4863-4807-9bc1-41bdf4fadb25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:17.471161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995fad0-4863-4807-9bc1-41bdf4fadb25 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:17.585045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 891/1769] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:17 2022] POST /identity/v3/users => generated 285 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:17.589668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8426f-36f2-43dc-aad1-744466d0419d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:17.601534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8426f-36f2-43dc-aad1-744466d0419d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:17.616931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0e1f8-6036-492b-984f-3c549899d5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:17.617848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0e1f8-6036-492b-984f-3c549899d5b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:17.618326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0e1f8-6036-492b-984f-3c549899d5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:17.618760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0e1f8-6036-492b-984f-3c549899d5b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:17.620344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b0e1f8-6036-492b-984f-3c549899d5b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:17.623427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b0e1f8-6036-492b-984f-3c549899d5b5 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:17.641745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 879/1770] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:17.645798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995fad0-4863-4807-9bc1-41bdf4fadb25 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:17.661462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995fad0-4863-4807-9bc1-41bdf4fadb25 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:17.667015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf11cb-8014-4938-ab32-98b0ea4a35a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:17.667708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf11cb-8014-4938-ab32-98b0ea4a35a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:17.667991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf11cb-8014-4938-ab32-98b0ea4a35a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:17.671154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf11cb-8014-4938-ab32-98b0ea4a35a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:17.671154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf11cb-8014-4938-ab32-98b0ea4a35a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:17.671154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf11cb-8014-4938-ab32-98b0ea4a35a2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:17.684980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 892/1771] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:44:17.689423 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0e1f8-6036-492b-984f-3c549899d5b5 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:17.712475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0e1f8-6036-492b-984f-3c549899d5b5 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:17.718203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9a96-6a48-44ab-b2ea-44a6a9ac831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:17.719470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9a96-6a48-44ab-b2ea-44a6a9ac831c admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:17.719853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9a96-6a48-44ab-b2ea-44a6a9ac831c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:17.720231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9a96-6a48-44ab-b2ea-44a6a9ac831c admin admin] PATH_INFO: `/v3/projects/00cd34c5788d4ac1a4bcff19d26f8d10/users/50c31cd004fb48e6b49901bb3885ff5c/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:17.738523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f9a96-6a48-44ab-b2ea-44a6a9ac831c admin admin] RBAC: Authorizing `identity:create_grant(project_id=00cd34c5788d4ac1a4bcff19d26f8d10, user_id=50c31cd004fb48e6b49901bb3885ff5c, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:17.741693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f9a96-6a48-44ab-b2ea-44a6a9ac831c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:17.897437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 880/1772] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:17 2022] PUT /identity/v3/projects/00cd34c5788d4ac1a4bcff19d26f8d10/users/50c31cd004fb48e6b49901bb3885ff5c/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:17.910904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da364d8-ec6c-483f-ad27-f68fac9f1cab None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:17.911611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da364d8-ec6c-483f-ad27-f68fac9f1cab None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:17.912060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da364d8-ec6c-483f-ad27-f68fac9f1cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:17.977505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5da364d8-ec6c-483f-ad27-f68fac9f1cab None None] MFA Rules not processed for user `50c31cd004fb48e6b49901bb3885ff5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:17.979385 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5da364d8-ec6c-483f-ad27-f68fac9f1cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:18.104391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 893/1773] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:18.105010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9a96-6a48-44ab-b2ea-44a6a9ac831c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:18.105010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-dd3f9a96-6a48-44ab-b2ea-44a6a9ac831c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:18.216276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9a96-6a48-44ab-b2ea-44a6a9ac831c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:18.220480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907922e-13f6-4c07-bec3-0c7795038451 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1a7213cbf6448b7ba6570a52a4c7c0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:18.221135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907922e-13f6-4c07-bec3-0c7795038451 service glance] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:18.221462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907922e-13f6-4c07-bec3-0c7795038451 service glance] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:18.222097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907922e-13f6-4c07-bec3-0c7795038451 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:18.231792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907922e-13f6-4c07-bec3-0c7795038451 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:18.233430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907922e-13f6-4c07-bec3-0c7795038451 service glance] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:18.244004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 881/1774] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:19.052129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:19.180793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf2f88-4d34-4d98-a843-2c1d6fc0bbed None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:19.183758 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf2f88-4d34-4d98-a843-2c1d6fc0bbed None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:19.184138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf2f88-4d34-4d98-a843-2c1d6fc0bbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:19.220485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1fcf2f88-4d34-4d98-a843-2c1d6fc0bbed None None] MFA Rules not processed for user `899cbac79d874b308b500651bb40b2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:19.221992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1fcf2f88-4d34-4d98-a843-2c1d6fc0bbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:19.288744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 894/1775] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:19.435982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254670e-04c0-4860-a3fa-653d538d2bd1 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:19.436635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254670e-04c0-4860-a3fa-653d538d2bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:19.436836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254670e-04c0-4860-a3fa-653d538d2bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:19.479212 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-d254670e-04c0-4860-a3fa-653d538d2bd1 None None] MFA Rules not processed for user `8773c88f88614a9d903f667305a217e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:19.481800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d254670e-04c0-4860-a3fa-653d538d2bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:19.522445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:19.568191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 882/1776] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:20.355976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d37603-f31f-4857-8e54-ba8dec6db33c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:20.355976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d37603-f31f-4857-8e54-ba8dec6db33c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:20.355976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d37603-f31f-4857-8e54-ba8dec6db33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:20.501791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d6d37603-f31f-4857-8e54-ba8dec6db33c None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:20.503132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d6d37603-f31f-4857-8e54-ba8dec6db33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:20.821325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 895/1777] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:44:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:20.832981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e6b32-b359-4099-97ca-31a6aa58d868 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:20.833576 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e6b32-b359-4099-97ca-31a6aa58d868 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:20.833764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e6b32-b359-4099-97ca-31a6aa58d868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:20.885510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-1f4e6b32-b359-4099-97ca-31a6aa58d868 None None] MFA Rules not processed for user `481ef56b1cfa465a8efc59736a34e412`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:20.896441 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1f4e6b32-b359-4099-97ca-31a6aa58d868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:21.023830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 883/1778] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:44:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:21.043640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d37603-f31f-4857-8e54-ba8dec6db33c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:21.045048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d6d37603-f31f-4857-8e54-ba8dec6db33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:21.069060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d37603-f31f-4857-8e54-ba8dec6db33c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:21.080452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbe0ac-d8ba-4acf-9d84-01bf060f81f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:21.081389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe0ac-d8ba-4acf-9d84-01bf060f81f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:21.081824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe0ac-d8ba-4acf-9d84-01bf060f81f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:21.082258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe0ac-d8ba-4acf-9d84-01bf060f81f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:21.102119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbe0ac-d8ba-4acf-9d84-01bf060f81f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:21.104458 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbe0ac-d8ba-4acf-9d84-01bf060f81f7 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:21.120654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 896/1779] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:44:21.387502 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e6b32-b359-4099-97ca-31a6aa58d868 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:21.391389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1f4e6b32-b359-4099-97ca-31a6aa58d868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:21.598283 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e6b32-b359-4099-97ca-31a6aa58d868 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:21.605326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5496c-e30a-43c8-a9e0-210776f6cd17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:21.606216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5496c-e30a-43c8-a9e0-210776f6cd17 service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:21.606646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5496c-e30a-43c8-a9e0-210776f6cd17 service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:21.607228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5496c-e30a-43c8-a9e0-210776f6cd17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:21.618239 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5496c-e30a-43c8-a9e0-210776f6cd17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:21.620064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5496c-e30a-43c8-a9e0-210776f6cd17 service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:21.629103 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 884/1780] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:44:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:44:21.651637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce1193-48c1-416e-9640-8156f1d2010f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:21.653228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce1193-48c1-416e-9640-8156f1d2010f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:21.653511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce1193-48c1-416e-9640-8156f1d2010f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:21.716076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a2ce1193-48c1-416e-9640-8156f1d2010f None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:21.717461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a2ce1193-48c1-416e-9640-8156f1d2010f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:21.729281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 897/1781] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:44:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:21.746936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5496c-e30a-43c8-a9e0-210776f6cd17 service swift] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:21.776424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5496c-e30a-43c8-a9e0-210776f6cd17 service swift] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:21.813965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e37a6d-3ba9-435f-aff0-1b7de73016dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:21.820143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e37a6d-3ba9-435f-aff0-1b7de73016dc service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:21.820462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e37a6d-3ba9-435f-aff0-1b7de73016dc service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:21.825532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e37a6d-3ba9-435f-aff0-1b7de73016dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:21.871492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e37a6d-3ba9-435f-aff0-1b7de73016dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:21.883092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e37a6d-3ba9-435f-aff0-1b7de73016dc service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:21.950056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 885/1782] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:44:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:44:22.034689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:22.296059 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:23.455405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:23.828737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce1193-48c1-416e-9640-8156f1d2010f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:23.831422 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a2ce1193-48c1-416e-9640-8156f1d2010f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:24.045706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce1193-48c1-416e-9640-8156f1d2010f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:24.051531 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:24.053058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:24.053559 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:24.054248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] PATH_INFO: `/v3/users/27be9eeb8e6c4065b60c694f58484def` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:24.056116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27be9eeb8e6c4065b60c694f58484def)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:24.058411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:24.278680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27be9eeb8e6c4065b60c694f58484def'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:24.304713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27be9eeb8e6c4065b60c694f58484def'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:24.319764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:24.330325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27be9eeb8e6c4065b60c694f58484def'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:24.433587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27be9eeb8e6c4065b60c694f58484def'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:24.434704 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27be9eeb8e6c4065b60c694f58484def'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:24.457927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 898/1783] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:23 2022] DELETE /identity/v3/users/27be9eeb8e6c4065b60c694f58484def => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:24.458095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905626d7-b9eb-476d-8f4a-a69a1793ffd5 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:24.458095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905626d7-b9eb-476d-8f4a-a69a1793ffd5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:24.458095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905626d7-b9eb-476d-8f4a-a69a1793ffd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:24.467312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:24.498365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-905626d7-b9eb-476d-8f4a-a69a1793ffd5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:24.499819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-905626d7-b9eb-476d-8f4a-a69a1793ffd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:24.671770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 886/1784] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:24.682036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:24.689572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:24.852143 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2191d89-ac2b-4c99-b45d-02c4c9d15e57 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:24.858756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f644e6-9515-4af4-8471-fa6bf326c656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:24.859545 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f644e6-9515-4af4-8471-fa6bf326c656 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:24.860140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f644e6-9515-4af4-8471-fa6bf326c656 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:24.860592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f644e6-9515-4af4-8471-fa6bf326c656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:24.876921 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f644e6-9515-4af4-8471-fa6bf326c656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:24.879361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f644e6-9515-4af4-8471-fa6bf326c656 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:24.915757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 899/1785] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:25.163824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905626d7-b9eb-476d-8f4a-a69a1793ffd5 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:25.165241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-905626d7-b9eb-476d-8f4a-a69a1793ffd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:25.190673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905626d7-b9eb-476d-8f4a-a69a1793ffd5 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:25.211184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:25.211184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:25.211184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:25.211184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] PATH_INFO: `/v3/users/9c8674cd6707429e84db02796f3c0db9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:25.211686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8674cd6707429e84db02796f3c0db9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:25.211686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:25.459918 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8674cd6707429e84db02796f3c0db9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:25.460645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8674cd6707429e84db02796f3c0db9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:25.479067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8674cd6707429e84db02796f3c0db9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:25.506796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8674cd6707429e84db02796f3c0db9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:25.639647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9782eb6c-9a37-48ce-992b-3b2402b14026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8674cd6707429e84db02796f3c0db9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:25.640096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f644e6-9515-4af4-8471-fa6bf326c656 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:25.641729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-32f644e6-9515-4af4-8471-fa6bf326c656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:25.654247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 887/1786] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:25 2022] DELETE /identity/v3/users/9c8674cd6707429e84db02796f3c0db9 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:25.663940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edc709-020e-4d65-8696-8b8ed40c1473 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:25.666688 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edc709-020e-4d65-8696-8b8ed40c1473 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:25.667006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edc709-020e-4d65-8696-8b8ed40c1473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:25.684844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f644e6-9515-4af4-8471-fa6bf326c656 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:25.705817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-20edc709-020e-4d65-8696-8b8ed40c1473 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:25.707339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-20edc709-020e-4d65-8696-8b8ed40c1473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:25.832147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:25.843748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:25.844053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:25.844337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] PATH_INFO: `/v3/projects/ef766347aba243a498aeae29dce0a68f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:25.849057 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef766347aba243a498aeae29dce0a68f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:25.863972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:25.875924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 888/1787] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:25.888382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edc709-020e-4d65-8696-8b8ed40c1473 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:25.891852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-20edc709-020e-4d65-8696-8b8ed40c1473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:25.990901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edc709-020e-4d65-8696-8b8ed40c1473 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:25.995089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a258b0-4304-42c6-9b42-1fc2a693f24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:25.995702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a258b0-4304-42c6-9b42-1fc2a693f24b service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:25.995990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a258b0-4304-42c6-9b42-1fc2a693f24b service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:25.996269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a258b0-4304-42c6-9b42-1fc2a693f24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:26.012182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a258b0-4304-42c6-9b42-1fc2a693f24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:26.015509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a258b0-4304-42c6-9b42-1fc2a693f24b service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:26.025824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 889/1788] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:26.031633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef766347aba243a498aeae29dce0a68f', 'request_id': 'req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.120096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef766347aba243a498aeae29dce0a68f', 'request_id': 'req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.127498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef766347aba243a498aeae29dce0a68f', 'request_id': 'req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.151861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] The token cache is being invalidate because project ef766347aba243a498aeae29dce0a68f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:26.152168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.154729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.156843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 900/1789] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:25 2022] DELETE /identity/v3/projects/ef766347aba243a498aeae29dce0a68f => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:26.160721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a258b0-4304-42c6-9b42-1fc2a693f24b service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:26.164774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d4a258b0-4304-42c6-9b42-1fc2a693f24b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:26.181748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a258b0-4304-42c6-9b42-1fc2a693f24b service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:26.270316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:26.271461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:26.272024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:26.272452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] PATH_INFO: `/v3/users/92d19188fbf14ded98794aa92ad6f560` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:26.275770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=92d19188fbf14ded98794aa92ad6f560)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:26.278688 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:26.467520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:26.567342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:26.568859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:26.579731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92d19188fbf14ded98794aa92ad6f560'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.580448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92d19188fbf14ded98794aa92ad6f560'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.591838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85cb1b-cf57-4fa2-b47d-ba762751d44d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:26.597390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92d19188fbf14ded98794aa92ad6f560'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.611133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:26.612767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:26.612767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:26.612767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] PATH_INFO: `/v3/projects/35423dbae50241b1b07df9d84b849588` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:26.614158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] RBAC: Authorizing `identity:delete_project(project_id=35423dbae50241b1b07df9d84b849588)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:26.616079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:26.644874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92d19188fbf14ded98794aa92ad6f560'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.771401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92d19188fbf14ded98794aa92ad6f560'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.810851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 890/1790] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:26 2022] DELETE /identity/v3/users/92d19188fbf14ded98794aa92ad6f560 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:26.890476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35423dbae50241b1b07df9d84b849588', 'request_id': 'req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:26.998661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35423dbae50241b1b07df9d84b849588', 'request_id': 'req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:27.012533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35423dbae50241b1b07df9d84b849588', 'request_id': 'req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:27.061313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] The token cache is being invalidate because project 35423dbae50241b1b07df9d84b849588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:27.061840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:27.064716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:27.069658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 901/1791] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:26 2022] DELETE /identity/v3/projects/35423dbae50241b1b07df9d84b849588 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:27.074066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:27.075717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:27.084713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:27.217395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08373ec3-3b4c-47fd-a676-92456ad9d4ea admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:27.222625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:27.224949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:27.224949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:27.224949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] PATH_INFO: `/v3/users/9190fcc540be4dc7abafe41ddf21fce2` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:27.226243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9190fcc540be4dc7abafe41ddf21fce2)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:27.229095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:27.275136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:27.488478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9190fcc540be4dc7abafe41ddf21fce2'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:27.489666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9190fcc540be4dc7abafe41ddf21fce2'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:27.510958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9190fcc540be4dc7abafe41ddf21fce2'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:27.521949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9190fcc540be4dc7abafe41ddf21fce2'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:27.615506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-330f65c8-d9e4-4ce1-a998-393ec109f07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9190fcc540be4dc7abafe41ddf21fce2'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:27.630782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 891/1792] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:27 2022] DELETE /identity/v3/users/9190fcc540be4dc7abafe41ddf21fce2 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:27.819451 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:27.821309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:27.986088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e4ae9-4aba-4bc0-b0e7-fdfc42a7109e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:28.106474 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:28.107667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:28.108110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:28.108529 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] PATH_INFO: `/v3/projects/8066f7e6b407475080059679d0c0b488` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:28.110230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8066f7e6b407475080059679d0c0b488)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:28.112476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:28.340640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8066f7e6b407475080059679d0c0b488', 'request_id': 'req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:28.400260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:28.454765 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8066f7e6b407475080059679d0c0b488', 'request_id': 'req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:28.469584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8066f7e6b407475080059679d0c0b488', 'request_id': 'req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:28.486333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] The token cache is being invalidate because project 8066f7e6b407475080059679d0c0b488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:28.486717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:28.487468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:28.492654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 902/1793] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:27 2022] DELETE /identity/v3/projects/8066f7e6b407475080059679d0c0b488 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:28.503183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b57e23-f4f4-4be2-b5b7-a5a0702d9d84 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:28.503183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b57e23-f4f4-4be2-b5b7-a5a0702d9d84 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:28.503183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b57e23-f4f4-4be2-b5b7-a5a0702d9d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:28.524895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-83b57e23-f4f4-4be2-b5b7-a5a0702d9d84 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:28.526182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-83b57e23-f4f4-4be2-b5b7-a5a0702d9d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:28.570692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:28.577401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:28.620820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 892/1794] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:28.640818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6052f-e836-4d93-822f-caab4d7c0241 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:28.641110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6052f-e836-4d93-822f-caab4d7c0241 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:28.645544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6052f-e836-4d93-822f-caab4d7c0241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:28.673118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-bbb6052f-e836-4d93-822f-caab4d7c0241 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:28.676706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-bbb6052f-e836-4d93-822f-caab4d7c0241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:28.685995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 893/1795] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:28.690479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6052f-e836-4d93-822f-caab4d7c0241 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:28.701440 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6052f-e836-4d93-822f-caab4d7c0241 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:28.705889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb0882-8659-4a2f-a43d-0a4bfb2a0085 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:28.709598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37155bb8-b99e-4c33-bfca-eb32ad423656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:28.710635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:28.710809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37155bb8-b99e-4c33-bfca-eb32ad423656 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:28.711244 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37155bb8-b99e-4c33-bfca-eb32ad423656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:28.711487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:28.711770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:28.712046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] PATH_INFO: `/v3/projects/0abd60efd2e04d66bc5ebc99341f9c8d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:28.712264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37155bb8-b99e-4c33-bfca-eb32ad423656 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:28.713356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37155bb8-b99e-4c33-bfca-eb32ad423656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:28.713486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0abd60efd2e04d66bc5ebc99341f9c8d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:28.713924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37155bb8-b99e-4c33-bfca-eb32ad423656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:28.715416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:28.716095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37155bb8-b99e-4c33-bfca-eb32ad423656 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:28.730456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 894/1796] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:44:28.735259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37155bb8-b99e-4c33-bfca-eb32ad423656 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:28.746135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37155bb8-b99e-4c33-bfca-eb32ad423656 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:28.751290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b4859-2e77-4928-b14b-5384a17e7ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:28.752006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b4859-2e77-4928-b14b-5384a17e7ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:28.752260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b4859-2e77-4928-b14b-5384a17e7ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:28.756135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b4859-2e77-4928-b14b-5384a17e7ad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:28.756135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b4859-2e77-4928-b14b-5384a17e7ad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:28.756135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b4859-2e77-4928-b14b-5384a17e7ad7 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:28.899476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0abd60efd2e04d66bc5ebc99341f9c8d', 'request_id': 'req-8e72c482-b267-4d55-8e41-f3be80b4dbd4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:28.917910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 895/1797] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:28 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:44:28.919074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b4859-2e77-4928-b14b-5384a17e7ad7 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:28.988085 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b4859-2e77-4928-b14b-5384a17e7ad7 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:28.992935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e16183-2336-4d46-af2f-bb3d2fb57d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:28.993643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e16183-2336-4d46-af2f-bb3d2fb57d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:28.993926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e16183-2336-4d46-af2f-bb3d2fb57d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:28.994224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e16183-2336-4d46-af2f-bb3d2fb57d8d admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:28.994941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e16183-2336-4d46-af2f-bb3d2fb57d8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:28.996888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e16183-2336-4d46-af2f-bb3d2fb57d8d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.009550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0abd60efd2e04d66bc5ebc99341f9c8d', 'request_id': 'req-8e72c482-b267-4d55-8e41-f3be80b4dbd4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:29.016438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0abd60efd2e04d66bc5ebc99341f9c8d', 'request_id': 'req-8e72c482-b267-4d55-8e41-f3be80b4dbd4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:29.033845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] The token cache is being invalidate because project 0abd60efd2e04d66bc5ebc99341f9c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:29.034241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:29.034815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8e72c482-b267-4d55-8e41-f3be80b4dbd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:29.037325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 903/1798] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:28 2022] DELETE /identity/v3/projects/0abd60efd2e04d66bc5ebc99341f9c8d => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:29.042221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d6810-78dc-46da-b3b9-4a78932b1a1b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.042511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d6810-78dc-46da-b3b9-4a78932b1a1b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.042792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d6810-78dc-46da-b3b9-4a78932b1a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.064054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ff8d6810-78dc-46da-b3b9-4a78932b1a1b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:29.065425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ff8d6810-78dc-46da-b3b9-4a78932b1a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:29.076030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 904/1799] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:29.088482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8807-a2a3-4c33-a6dd-08e418e2da3b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.088878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8807-a2a3-4c33-a6dd-08e418e2da3b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.089263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8807-a2a3-4c33-a6dd-08e418e2da3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.112178 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b2ee8807-a2a3-4c33-a6dd-08e418e2da3b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:29.113449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b2ee8807-a2a3-4c33-a6dd-08e418e2da3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:29.116079 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 896/1800] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:28 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:44:29.117987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e16183-2336-4d46-af2f-bb3d2fb57d8d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.124594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-26e16183-2336-4d46-af2f-bb3d2fb57d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:29.134473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 905/1801] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:29.142785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee8807-a2a3-4c33-a6dd-08e418e2da3b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.162395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e16183-2336-4d46-af2f-bb3d2fb57d8d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.168863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5f832-5f41-4c10-ac57-97a2eff99cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.169751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f832-5f41-4c10-ac57-97a2eff99cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.170091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee8807-a2a3-4c33-a6dd-08e418e2da3b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.170701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f832-5f41-4c10-ac57-97a2eff99cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.171177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f832-5f41-4c10-ac57-97a2eff99cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.172737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5f832-5f41-4c10-ac57-97a2eff99cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.175002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5f832-5f41-4c10-ac57-97a2eff99cf1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.176418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b29c1-2493-4408-89d0-5cf4dc8de89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.177328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b29c1-2493-4408-89d0-5cf4dc8de89f admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.178156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b29c1-2493-4408-89d0-5cf4dc8de89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.179262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b29c1-2493-4408-89d0-5cf4dc8de89f admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.180121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b29c1-2493-4408-89d0-5cf4dc8de89f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:29.180509 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b29c1-2493-4408-89d0-5cf4dc8de89f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.182589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b29c1-2493-4408-89d0-5cf4dc8de89f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.193935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 897/1802] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:29.198476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5f832-5f41-4c10-ac57-97a2eff99cf1 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.206967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 906/1803] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:29.208085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b29c1-2493-4408-89d0-5cf4dc8de89f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.213506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5f832-5f41-4c10-ac57-97a2eff99cf1 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.219648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b29c1-2493-4408-89d0-5cf4dc8de89f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.223594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e5d77-0a69-4d61-9e59-ac702223dc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.223594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e5d77-0a69-4d61-9e59-ac702223dc18 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.223594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e5d77-0a69-4d61-9e59-ac702223dc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.223594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e5d77-0a69-4d61-9e59-ac702223dc18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.223971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e5d77-0a69-4d61-9e59-ac702223dc18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.224568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e5d77-0a69-4d61-9e59-ac702223dc18 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.225428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bb0c7-fe1e-41e8-a85c-11c2c73ef2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.226324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bb0c7-fe1e-41e8-a85c-11c2c73ef2eb admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.226780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bb0c7-fe1e-41e8-a85c-11c2c73ef2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.227054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bb0c7-fe1e-41e8-a85c-11c2c73ef2eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.228011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bb0c7-fe1e-41e8-a85c-11c2c73ef2eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.230251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bb0c7-fe1e-41e8-a85c-11c2c73ef2eb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.236553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 898/1804] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:44:29.240862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e5d77-0a69-4d61-9e59-ac702223dc18 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.253450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e5d77-0a69-4d61-9e59-ac702223dc18 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.259177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772eeda-99f3-4e2c-8c1a-cbd5e53e2d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.260846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772eeda-99f3-4e2c-8c1a-cbd5e53e2d73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.261133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772eeda-99f3-4e2c-8c1a-cbd5e53e2d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.261454 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772eeda-99f3-4e2c-8c1a-cbd5e53e2d73 admin admin] PATH_INFO: `/v3/projects/d0f42396eba94008941524a931b3549a/users/7e0b1f173f8a4e7dbee8874f140d8877/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.274264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:29.280149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772eeda-99f3-4e2c-8c1a-cbd5e53e2d73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f42396eba94008941524a931b3549a, user_id=7e0b1f173f8a4e7dbee8874f140d8877, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.283148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772eeda-99f3-4e2c-8c1a-cbd5e53e2d73 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.326254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 907/1805] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:29 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:29.328609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bb0c7-fe1e-41e8-a85c-11c2c73ef2eb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.372196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 899/1806] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:29 2022] PUT /identity/v3/projects/d0f42396eba94008941524a931b3549a/users/7e0b1f173f8a4e7dbee8874f140d8877/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:29.382777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f40f3-472f-4d65-9c47-33e0f824f837 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.383072 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f40f3-472f-4d65-9c47-33e0f824f837 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.383359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f40f3-472f-4d65-9c47-33e0f824f837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.427987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-093f40f3-472f-4d65-9c47-33e0f824f837 None None] MFA Rules not processed for user `7e0b1f173f8a4e7dbee8874f140d8877`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:29.429263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-093f40f3-472f-4d65-9c47-33e0f824f837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:29.435251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bb0c7-fe1e-41e8-a85c-11c2c73ef2eb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.545659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 900/1807] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:29.547347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f40f3-472f-4d65-9c47-33e0f824f837 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.551982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022b838-8e0d-42a6-9b49-d4c0369a68f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.552681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022b838-8e0d-42a6-9b49-d4c0369a68f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.552966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022b838-8e0d-42a6-9b49-d4c0369a68f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.553323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022b838-8e0d-42a6-9b49-d4c0369a68f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.554267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022b838-8e0d-42a6-9b49-d4c0369a68f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.557444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022b838-8e0d-42a6-9b49-d4c0369a68f7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.559141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f40f3-472f-4d65-9c47-33e0f824f837 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.564202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b14a70-bda2-432a-99a8-9c2c8127b95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.564952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b14a70-bda2-432a-99a8-9c2c8127b95c admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.566301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b14a70-bda2-432a-99a8-9c2c8127b95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.566723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b14a70-bda2-432a-99a8-9c2c8127b95c admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.567526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b14a70-bda2-432a-99a8-9c2c8127b95c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.569739 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b14a70-bda2-432a-99a8-9c2c8127b95c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.574456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:29.662653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 908/1808] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:29 2022] POST /identity/v3/users => generated 284 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:44:29.667002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022b838-8e0d-42a6-9b49-d4c0369a68f7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.679502 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 901/1809] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:29 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:29.684608 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b14a70-bda2-432a-99a8-9c2c8127b95c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.686287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022b838-8e0d-42a6-9b49-d4c0369a68f7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.779365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b14a70-bda2-432a-99a8-9c2c8127b95c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.785272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299aa74-3e6f-4178-90f8-f4f8ef8b3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.786177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299aa74-3e6f-4178-90f8-f4f8ef8b3e54 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.786660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299aa74-3e6f-4178-90f8-f4f8ef8b3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.786765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299aa74-3e6f-4178-90f8-f4f8ef8b3e54 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.787460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299aa74-3e6f-4178-90f8-f4f8ef8b3e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.789461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299aa74-3e6f-4178-90f8-f4f8ef8b3e54 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.792976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59c83c-a946-4ae3-800f-37a2e63f82bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.793702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59c83c-a946-4ae3-800f-37a2e63f82bb admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.794091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59c83c-a946-4ae3-800f-37a2e63f82bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.794469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59c83c-a946-4ae3-800f-37a2e63f82bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.795267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59c83c-a946-4ae3-800f-37a2e63f82bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.797204 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59c83c-a946-4ae3-800f-37a2e63f82bb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.813205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 909/1810] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:29.816295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59c83c-a946-4ae3-800f-37a2e63f82bb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.826768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59c83c-a946-4ae3-800f-37a2e63f82bb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.831437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d03689-0ace-41c1-b173-f707721cd324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.832104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d03689-0ace-41c1-b173-f707721cd324 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.832380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d03689-0ace-41c1-b173-f707721cd324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.832653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d03689-0ace-41c1-b173-f707721cd324 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.833456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d03689-0ace-41c1-b173-f707721cd324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.835122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d03689-0ace-41c1-b173-f707721cd324 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.847093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 910/1811] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:44:29.852046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d03689-0ace-41c1-b173-f707721cd324 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.866947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d03689-0ace-41c1-b173-f707721cd324 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.874819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab63fc-6229-41b8-929d-8b84d5e351c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.876646 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab63fc-6229-41b8-929d-8b84d5e351c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.876963 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab63fc-6229-41b8-929d-8b84d5e351c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.877395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab63fc-6229-41b8-929d-8b84d5e351c2 admin admin] PATH_INFO: `/v3/projects/63659efb2a904d348ec7795a11d31edc/users/1db04445458d4489b8085cbfd58d3c3b/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.898944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab63fc-6229-41b8-929d-8b84d5e351c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63659efb2a904d348ec7795a11d31edc, user_id=1db04445458d4489b8085cbfd58d3c3b, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.902310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab63fc-6229-41b8-929d-8b84d5e351c2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.911769 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 902/1812] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:29 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:44:29.920425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299aa74-3e6f-4178-90f8-f4f8ef8b3e54 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.931714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299aa74-3e6f-4178-90f8-f4f8ef8b3e54 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.938353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019a73e-a36c-4e04-8d29-6a1a8706f8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.939233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a73e-a36c-4e04-8d29-6a1a8706f8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.939661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a73e-a36c-4e04-8d29-6a1a8706f8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.939988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a73e-a36c-4e04-8d29-6a1a8706f8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:29.940772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019a73e-a36c-4e04-8d29-6a1a8706f8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:29.945614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019a73e-a36c-4e04-8d29-6a1a8706f8f9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:29.956196 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 903/1813] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:44:29.960555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019a73e-a36c-4e04-8d29-6a1a8706f8f9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:29.977979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019a73e-a36c-4e04-8d29-6a1a8706f8f9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:29.989367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188214d-c83e-4d5a-9b6d-99bd8ebb211b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:29.990920 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188214d-c83e-4d5a-9b6d-99bd8ebb211b admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:29.991406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188214d-c83e-4d5a-9b6d-99bd8ebb211b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:29.992192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188214d-c83e-4d5a-9b6d-99bd8ebb211b admin admin] PATH_INFO: `/v3/projects/92ab4014d5d74de49da5658ec5d6ae3d/users/7251daf3d8564496af2d3f4278c87b6b/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:30.013456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188214d-c83e-4d5a-9b6d-99bd8ebb211b admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ab4014d5d74de49da5658ec5d6ae3d, user_id=7251daf3d8564496af2d3f4278c87b6b, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:30.016280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188214d-c83e-4d5a-9b6d-99bd8ebb211b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:30.025402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 911/1814] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:29 2022] PUT /identity/v3/projects/63659efb2a904d348ec7795a11d31edc/users/1db04445458d4489b8085cbfd58d3c3b/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:30.040151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29a6d0-d0bc-459c-881f-bde34b391522 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:30.040515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29a6d0-d0bc-459c-881f-bde34b391522 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:30.041607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29a6d0-d0bc-459c-881f-bde34b391522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:30.086017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0c29a6d0-d0bc-459c-881f-bde34b391522 None None] MFA Rules not processed for user `1db04445458d4489b8085cbfd58d3c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:30.087728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0c29a6d0-d0bc-459c-881f-bde34b391522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:30.119021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 904/1815] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:29 2022] PUT /identity/v3/projects/92ab4014d5d74de49da5658ec5d6ae3d/users/7251daf3d8564496af2d3f4278c87b6b/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:30.123433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188214d-c83e-4d5a-9b6d-99bd8ebb211b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:30.173311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 912/1816] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:30.180158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29a6d0-d0bc-459c-881f-bde34b391522 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:30.270324 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188214d-c83e-4d5a-9b6d-99bd8ebb211b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:30.276588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc534d-de4b-4b33-b4dd-5cc512d6999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:30.277685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc534d-de4b-4b33-b4dd-5cc512d6999d admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:30.278347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc534d-de4b-4b33-b4dd-5cc512d6999d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:30.278851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc534d-de4b-4b33-b4dd-5cc512d6999d admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:30.279893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc534d-de4b-4b33-b4dd-5cc512d6999d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:30.282165 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc534d-de4b-4b33-b4dd-5cc512d6999d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:30.296292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 905/1817] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:44:30.301298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29a6d0-d0bc-459c-881f-bde34b391522 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:30.302679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc534d-de4b-4b33-b4dd-5cc512d6999d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:30.315658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc534d-de4b-4b33-b4dd-5cc512d6999d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:30.320884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3ee3f-e6f8-4ab3-8550-c0169c368528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:30.321248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3ee3f-e6f8-4ab3-8550-c0169c368528 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:30.321518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3ee3f-e6f8-4ab3-8550-c0169c368528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:30.321796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3ee3f-e6f8-4ab3-8550-c0169c368528 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:30.322641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3ee3f-e6f8-4ab3-8550-c0169c368528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:30.324654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e8fef-ab0e-4e8c-99be-ab0ab6f79706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:30.326410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e8fef-ab0e-4e8c-99be-ab0ab6f79706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:30.326906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3ee3f-e6f8-4ab3-8550-c0169c368528 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:30.329535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e8fef-ab0e-4e8c-99be-ab0ab6f79706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:30.330108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e8fef-ab0e-4e8c-99be-ab0ab6f79706 admin admin] PATH_INFO: `/v3/projects/92ab4014d5d74de49da5658ec5d6ae3d/users/7251daf3d8564496af2d3f4278c87b6b/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:30.333411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e8fef-ab0e-4e8c-99be-ab0ab6f79706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ab4014d5d74de49da5658ec5d6ae3d, user_id=7251daf3d8564496af2d3f4278c87b6b, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:30.336741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e8fef-ab0e-4e8c-99be-ab0ab6f79706 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:30.348676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 906/1818] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:30 2022] PUT /identity/v3/projects/92ab4014d5d74de49da5658ec5d6ae3d/users/7251daf3d8564496af2d3f4278c87b6b/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:30.359720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc25f49-4b31-4e77-a7a1-e63ed06ab43d None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:30.360031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc25f49-4b31-4e77-a7a1-e63ed06ab43d None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:30.360304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc25f49-4b31-4e77-a7a1-e63ed06ab43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:30.397286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-4cc25f49-4b31-4e77-a7a1-e63ed06ab43d None None] MFA Rules not processed for user `7251daf3d8564496af2d3f4278c87b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:30.398744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4cc25f49-4b31-4e77-a7a1-e63ed06ab43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:30.471051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 913/1819] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:30 2022] POST /identity/v3/projects => generated 367 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:44:30.475798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3ee3f-e6f8-4ab3-8550-c0169c368528 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:30.486157 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 907/1820] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:30.500796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc25f49-4b31-4e77-a7a1-e63ed06ab43d None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:30.513500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4cc25f49-4b31-4e77-a7a1-e63ed06ab43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:30.569980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3ee3f-e6f8-4ab3-8550-c0169c368528 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:30.586068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0f5ac-feb5-4e37-b0f6-f6803b848cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:30.587031 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0f5ac-feb5-4e37-b0f6-f6803b848cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:30.587444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0f5ac-feb5-4e37-b0f6-f6803b848cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:30.588402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0f5ac-feb5-4e37-b0f6-f6803b848cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:30.589278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0f5ac-feb5-4e37-b0f6-f6803b848cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:30.592024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0f5ac-feb5-4e37-b0f6-f6803b848cd2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:30.610345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc25f49-4b31-4e77-a7a1-e63ed06ab43d None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:30.679153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 914/1821] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:30 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:44:30.683477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12ed6-3532-488f-8999-2420d8e0dcec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:30.684161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12ed6-3532-488f-8999-2420d8e0dcec service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:30.684285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0f5ac-feb5-4e37-b0f6-f6803b848cd2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:30.684450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12ed6-3532-488f-8999-2420d8e0dcec service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:30.684699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12ed6-3532-488f-8999-2420d8e0dcec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:30.694961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0f5ac-feb5-4e37-b0f6-f6803b848cd2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:30.703489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374f33a-5e64-4910-8507-0205d31de447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:30.704183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374f33a-5e64-4910-8507-0205d31de447 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:30.704472 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374f33a-5e64-4910-8507-0205d31de447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:30.704753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374f33a-5e64-4910-8507-0205d31de447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:30.705669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374f33a-5e64-4910-8507-0205d31de447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:30.707342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374f33a-5e64-4910-8507-0205d31de447 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:30.723954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 915/1822] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:44:30.725018 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374f33a-5e64-4910-8507-0205d31de447 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:30.739513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374f33a-5e64-4910-8507-0205d31de447 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:30.745451 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57426957-2993-4db8-8d7c-25904dfcdba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:30.746413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57426957-2993-4db8-8d7c-25904dfcdba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:30.746697 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57426957-2993-4db8-8d7c-25904dfcdba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:30.746970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57426957-2993-4db8-8d7c-25904dfcdba6 admin admin] PATH_INFO: `/v3/projects/8df00e04f89445e887672557af2551fc/users/5310a53319c94ab49a5a5d757bdc9ad1/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:30.762517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf12ed6-3532-488f-8999-2420d8e0dcec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:30.763991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf12ed6-3532-488f-8999-2420d8e0dcec service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:30.778209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 908/1823] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:30.781649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57426957-2993-4db8-8d7c-25904dfcdba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df00e04f89445e887672557af2551fc, user_id=5310a53319c94ab49a5a5d757bdc9ad1, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:30.785824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57426957-2993-4db8-8d7c-25904dfcdba6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:30.787589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:30.911687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 916/1824] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:30 2022] PUT /identity/v3/projects/8df00e04f89445e887672557af2551fc/users/5310a53319c94ab49a5a5d757bdc9ad1/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:30.911881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12ed6-3532-488f-8999-2420d8e0dcec service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:30.926394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12ed6-3532-488f-8999-2420d8e0dcec service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:30.971664 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57426957-2993-4db8-8d7c-25904dfcdba6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:31.000685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07520abf-8c7a-45df-821c-8b1660717228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:31.001505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07520abf-8c7a-45df-821c-8b1660717228 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:31.001856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07520abf-8c7a-45df-821c-8b1660717228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:31.002198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07520abf-8c7a-45df-821c-8b1660717228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:31.003051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07520abf-8c7a-45df-821c-8b1660717228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:31.004716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07520abf-8c7a-45df-821c-8b1660717228 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:31.015477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 909/1825] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:31.019675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07520abf-8c7a-45df-821c-8b1660717228 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:31.030247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07520abf-8c7a-45df-821c-8b1660717228 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:31.035050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb984977-7196-4845-9dd3-148190d9706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:31.035926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984977-7196-4845-9dd3-148190d9706b admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:31.036205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984977-7196-4845-9dd3-148190d9706b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:31.036478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984977-7196-4845-9dd3-148190d9706b admin admin] PATH_INFO: `/v3/projects/8df00e04f89445e887672557af2551fc/users/5310a53319c94ab49a5a5d757bdc9ad1/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:31.038632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb984977-7196-4845-9dd3-148190d9706b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df00e04f89445e887672557af2551fc, user_id=5310a53319c94ab49a5a5d757bdc9ad1, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:31.038885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57426957-2993-4db8-8d7c-25904dfcdba6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:31.041313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb984977-7196-4845-9dd3-148190d9706b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:31.043488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:31.044506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:31.044784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:31.045241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] PATH_INFO: `/v3/users/7e0b1f173f8a4e7dbee8874f140d8877` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:31.046452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e0b1f173f8a4e7dbee8874f140d8877)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:31.048402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:31.055043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:31.056790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 910/1826] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:31 2022] PUT /identity/v3/projects/8df00e04f89445e887672557af2551fc/users/5310a53319c94ab49a5a5d757bdc9ad1/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:31.069766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bc0c7-8e80-4b39-a96c-c64188498538 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:31.070062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bc0c7-8e80-4b39-a96c-c64188498538 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:31.070347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bc0c7-8e80-4b39-a96c-c64188498538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:31.097309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-8c8bc0c7-8e80-4b39-a96c-c64188498538 None None] MFA Rules not processed for user `5310a53319c94ab49a5a5d757bdc9ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:31.098575 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8c8bc0c7-8e80-4b39-a96c-c64188498538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:31.197403 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 911/1827] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:31.201540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb97006-df1c-46bd-bafd-c2bfa8b84e96 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:31.201540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb97006-df1c-46bd-bafd-c2bfa8b84e96 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:31.201540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb97006-df1c-46bd-bafd-c2bfa8b84e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:31.219109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-feb97006-df1c-46bd-bafd-c2bfa8b84e96 None None] MFA Rules not processed for user `481ef56b1cfa465a8efc59736a34e412`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:31.222732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-feb97006-df1c-46bd-bafd-c2bfa8b84e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:31.238750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e0b1f173f8a4e7dbee8874f140d8877'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:31.246954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e0b1f173f8a4e7dbee8874f140d8877'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:31.259949 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e0b1f173f8a4e7dbee8874f140d8877'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:31.276975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 912/1828] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Jul 14 11:44:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:31.278068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb97006-df1c-46bd-bafd-c2bfa8b84e96 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:31.280264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-feb97006-df1c-46bd-bafd-c2bfa8b84e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:31.340530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb97006-df1c-46bd-bafd-c2bfa8b84e96 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:31.344598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13b3ca-b1d6-4121-b791-d328534504e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:31.346110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13b3ca-b1d6-4121-b791-d328534504e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:31.346110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13b3ca-b1d6-4121-b791-d328534504e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:31.346110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13b3ca-b1d6-4121-b791-d328534504e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:31.356175 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13b3ca-b1d6-4121-b791-d328534504e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:31.357655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13b3ca-b1d6-4121-b791-d328534504e0 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:31.366349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e0b1f173f8a4e7dbee8874f140d8877'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:31.367402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-78c4e7e9-0e53-4f02-9f47-7134a064729b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e0b1f173f8a4e7dbee8874f140d8877'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:31.368417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 913/1829] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:31.369979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13b3ca-b1d6-4121-b791-d328534504e0 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:31.371525 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9c13b3ca-b1d6-4121-b791-d328534504e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:31.385900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13b3ca-b1d6-4121-b791-d328534504e0 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:31.390011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d203a-137f-4c75-bc23-9e2af469c4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:31.390651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d203a-137f-4c75-bc23-9e2af469c4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:31.390989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d203a-137f-4c75-bc23-9e2af469c4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:31.391271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d203a-137f-4c75-bc23-9e2af469c4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:31.393298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b03d203a-137f-4c75-bc23-9e2af469c4c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:31.394626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 917/1830] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:30 2022] DELETE /identity/v3/users/7e0b1f173f8a4e7dbee8874f140d8877 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:31.399174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be4806-a3d5-4857-9cd7-f49fa98d447a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:31.399463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be4806-a3d5-4857-9cd7-f49fa98d447a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:31.403213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be4806-a3d5-4857-9cd7-f49fa98d447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:31.408682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d203a-137f-4c75-bc23-9e2af469c4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:31.410258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d203a-137f-4c75-bc23-9e2af469c4c4 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:31.420571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 914/1831] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:31.435617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-06be4806-a3d5-4857-9cd7-f49fa98d447a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:31.436859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-06be4806-a3d5-4857-9cd7-f49fa98d447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:31.533593 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 918/1832] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:31.536572 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d203a-137f-4c75-bc23-9e2af469c4c4 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:31.538057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b03d203a-137f-4c75-bc23-9e2af469c4c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:31.549636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d203a-137f-4c75-bc23-9e2af469c4c4 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:31.622119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07232ac0-030e-48f0-b982-c34d63f57a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:31.622768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07232ac0-030e-48f0-b982-c34d63f57a1c service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:31.623057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07232ac0-030e-48f0-b982-c34d63f57a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:31.623318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07232ac0-030e-48f0-b982-c34d63f57a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:31.635329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07232ac0-030e-48f0-b982-c34d63f57a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:31.637298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07232ac0-030e-48f0-b982-c34d63f57a1c service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:31.647968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 915/1833] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:31.704897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be4806-a3d5-4857-9cd7-f49fa98d447a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:31.706626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-06be4806-a3d5-4857-9cd7-f49fa98d447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:31.814020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be4806-a3d5-4857-9cd7-f49fa98d447a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:31.820486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03e890-36ef-49da-b8cd-bdd7949bb0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:31.821177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e890-36ef-49da-b8cd-bdd7949bb0dd service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:31.821484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e890-36ef-49da-b8cd-bdd7949bb0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:31.821757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e890-36ef-49da-b8cd-bdd7949bb0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:31.824632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7f03e890-36ef-49da-b8cd-bdd7949bb0dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:31.917776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03e890-36ef-49da-b8cd-bdd7949bb0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:31.919291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03e890-36ef-49da-b8cd-bdd7949bb0dd service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:31.932243 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 919/1834] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:32.746290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07232ac0-030e-48f0-b982-c34d63f57a1c service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:32.747849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-07232ac0-030e-48f0-b982-c34d63f57a1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:32.763401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07232ac0-030e-48f0-b982-c34d63f57a1c service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:32.768135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:32.769014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:32.769354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:32.769753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] PATH_INFO: `/v3/projects/d0f42396eba94008941524a931b3549a` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:32.771422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f42396eba94008941524a931b3549a)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:32.773740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:32.970540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:33.016277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f42396eba94008941524a931b3549a', 'request_id': 'req-a51d722f-971a-40e2-b094-7a1754d6138b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:33.022741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f42396eba94008941524a931b3549a', 'request_id': 'req-a51d722f-971a-40e2-b094-7a1754d6138b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:33.055061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f42396eba94008941524a931b3549a', 'request_id': 'req-a51d722f-971a-40e2-b094-7a1754d6138b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:33.147407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] The token cache is being invalidate because project d0f42396eba94008941524a931b3549a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:33.147709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:33.156607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a51d722f-971a-40e2-b094-7a1754d6138b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:33.163150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 916/1835] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:32 2022] DELETE /identity/v3/projects/d0f42396eba94008941524a931b3549a => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:33.164440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03e890-36ef-49da-b8cd-bdd7949bb0dd service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:33.183487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03e890-36ef-49da-b8cd-bdd7949bb0dd service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:33.331271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:33.333850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:33.334394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:33.334682 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] PATH_INFO: `/v3/users/7251daf3d8564496af2d3f4278c87b6b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:33.337001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7251daf3d8564496af2d3f4278c87b6b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:33.339176 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:33.729555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7251daf3d8564496af2d3f4278c87b6b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:33.736536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7251daf3d8564496af2d3f4278c87b6b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:33.748722 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7251daf3d8564496af2d3f4278c87b6b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:33.857680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7251daf3d8564496af2d3f4278c87b6b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:33.858878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-070d2e9f-ba01-44ca-a196-10dc6ccba229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7251daf3d8564496af2d3f4278c87b6b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:33.866104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 920/1836] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:33 2022] DELETE /identity/v3/users/7251daf3d8564496af2d3f4278c87b6b => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:33.943051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3368a80-a757-48c9-86ad-5c8c26339c14 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:33.943607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3368a80-a757-48c9-86ad-5c8c26339c14 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:33.944074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3368a80-a757-48c9-86ad-5c8c26339c14 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:33.946577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 917/1837] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:44:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:44:33.974489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25691279-6504-46cd-91d9-4b0d79274ac2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:33.974489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25691279-6504-46cd-91d9-4b0d79274ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:33.974489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25691279-6504-46cd-91d9-4b0d79274ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:34.017297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-25691279-6504-46cd-91d9-4b0d79274ac2 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:34.018985 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-25691279-6504-46cd-91d9-4b0d79274ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:34.122559 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 921/1838] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:44:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:34.378723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3368a80-a757-48c9-86ad-5c8c26339c14 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:34.381140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f3368a80-a757-48c9-86ad-5c8c26339c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:34.424483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3368a80-a757-48c9-86ad-5c8c26339c14 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:34.434827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f9940-dfc8-4476-b852-67cc5b62939c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:34.435643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9940-dfc8-4476-b852-67cc5b62939c service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:34.435958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9940-dfc8-4476-b852-67cc5b62939c service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:34.436214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9940-dfc8-4476-b852-67cc5b62939c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:34.464982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f9940-dfc8-4476-b852-67cc5b62939c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:34.470658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f9940-dfc8-4476-b852-67cc5b62939c service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:34.487688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:34.495443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 918/1839] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:44:35.292911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25691279-6504-46cd-91d9-4b0d79274ac2 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:35.294376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-25691279-6504-46cd-91d9-4b0d79274ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:35.443302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25691279-6504-46cd-91d9-4b0d79274ac2 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:35.451419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:35.452352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:35.452630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:35.452906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] PATH_INFO: `/v3/projects/92ab4014d5d74de49da5658ec5d6ae3d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:35.454506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ab4014d5d74de49da5658ec5d6ae3d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:35.456464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:35.613936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ab4014d5d74de49da5658ec5d6ae3d', 'request_id': 'req-721d405e-7def-47b2-8746-204113968c6c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:35.702338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ab4014d5d74de49da5658ec5d6ae3d', 'request_id': 'req-721d405e-7def-47b2-8746-204113968c6c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:35.710997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ab4014d5d74de49da5658ec5d6ae3d', 'request_id': 'req-721d405e-7def-47b2-8746-204113968c6c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:35.725252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] The token cache is being invalidate because project 92ab4014d5d74de49da5658ec5d6ae3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:35.725696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:35.726755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-721d405e-7def-47b2-8746-204113968c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:35.728756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 922/1840] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:35 2022] DELETE /identity/v3/projects/92ab4014d5d74de49da5658ec5d6ae3d => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:35.734828 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd582a-f76a-41ac-9e18-dbb01e876c0f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:35.735880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd582a-f76a-41ac-9e18-dbb01e876c0f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:35.736837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd582a-f76a-41ac-9e18-dbb01e876c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:35.787338 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-6fcd582a-f76a-41ac-9e18-dbb01e876c0f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:35.788620 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6fcd582a-f76a-41ac-9e18-dbb01e876c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:35.920374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 919/1841] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:35.929011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5219b-3a10-4ed2-8dea-c9436243673d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:35.930745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5219b-3a10-4ed2-8dea-c9436243673d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:35.930745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5219b-3a10-4ed2-8dea-c9436243673d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:35.992174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3cf5219b-3a10-4ed2-8dea-c9436243673d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:35.992174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3cf5219b-3a10-4ed2-8dea-c9436243673d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:36.053470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 923/1842] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:36.061227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd582a-f76a-41ac-9e18-dbb01e876c0f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:36.076258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd582a-f76a-41ac-9e18-dbb01e876c0f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:36.086490 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af227e-183d-4ea2-a838-94cd58678c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:36.087314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af227e-183d-4ea2-a838-94cd58678c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:36.087607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af227e-183d-4ea2-a838-94cd58678c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:36.087978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af227e-183d-4ea2-a838-94cd58678c26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:36.089216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af227e-183d-4ea2-a838-94cd58678c26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:36.089538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af227e-183d-4ea2-a838-94cd58678c26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:36.093422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af227e-183d-4ea2-a838-94cd58678c26 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:36.146159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 920/1843] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:36.146263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5219b-3a10-4ed2-8dea-c9436243673d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:36.163607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5219b-3a10-4ed2-8dea-c9436243673d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:36.186168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0568c9-3a28-4573-906a-59a3586bd418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:36.198793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0568c9-3a28-4573-906a-59a3586bd418 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:36.199343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0568c9-3a28-4573-906a-59a3586bd418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:36.199697 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0568c9-3a28-4573-906a-59a3586bd418 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:36.200500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0568c9-3a28-4573-906a-59a3586bd418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:36.202572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0568c9-3a28-4573-906a-59a3586bd418 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:36.411192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 924/1844] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:36 2022] POST /identity/v3/projects => generated 379 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:36.415411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af227e-183d-4ea2-a838-94cd58678c26 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:36.427592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:36.434358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af227e-183d-4ea2-a838-94cd58678c26 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:36.520004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bb23b-87b6-46fb-bdcd-3cd3b1e9298b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:36.520925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bb23b-87b6-46fb-bdcd-3cd3b1e9298b admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:36.521439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bb23b-87b6-46fb-bdcd-3cd3b1e9298b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:36.521863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bb23b-87b6-46fb-bdcd-3cd3b1e9298b admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:36.522905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bb23b-87b6-46fb-bdcd-3cd3b1e9298b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:36.531642 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bb23b-87b6-46fb-bdcd-3cd3b1e9298b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:36.655743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 921/1845] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:36 2022] POST /identity/v3/users => generated 290 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:36.659625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0568c9-3a28-4573-906a-59a3586bd418 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:36.682060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0568c9-3a28-4573-906a-59a3586bd418 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:36.703528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27808225-76b7-4f55-a10b-f733959c403e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:36.704340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808225-76b7-4f55-a10b-f733959c403e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:36.706142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808225-76b7-4f55-a10b-f733959c403e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:36.706508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808225-76b7-4f55-a10b-f733959c403e admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:36.708454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27808225-76b7-4f55-a10b-f733959c403e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:36.710279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27808225-76b7-4f55-a10b-f733959c403e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:36.727766 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 925/1846] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:36.729044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bb23b-87b6-46fb-bdcd-3cd3b1e9298b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:36.739456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bb23b-87b6-46fb-bdcd-3cd3b1e9298b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:36.754692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809edf2b-aa0a-4d5c-bcc8-ec495a11a2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:36.756033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809edf2b-aa0a-4d5c-bcc8-ec495a11a2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:36.756495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809edf2b-aa0a-4d5c-bcc8-ec495a11a2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:36.757024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809edf2b-aa0a-4d5c-bcc8-ec495a11a2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:36.758001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809edf2b-aa0a-4d5c-bcc8-ec495a11a2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:36.760126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809edf2b-aa0a-4d5c-bcc8-ec495a11a2b3 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:36.786767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 922/1847] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:36.791952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27808225-76b7-4f55-a10b-f733959c403e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:36.802553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27808225-76b7-4f55-a10b-f733959c403e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:36.824817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb40126-3974-4290-bd91-cafec8cbaa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:36.828513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb40126-3974-4290-bd91-cafec8cbaa14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:36.828807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb40126-3974-4290-bd91-cafec8cbaa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:36.829342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb40126-3974-4290-bd91-cafec8cbaa14 admin admin] PATH_INFO: `/v3/projects/1b896eb810a04025b17fff33697fefe8/users/05f29ccf27fa4030b0dbc751a8ca9c35/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:36.857128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb40126-3974-4290-bd91-cafec8cbaa14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b896eb810a04025b17fff33697fefe8, user_id=05f29ccf27fa4030b0dbc751a8ca9c35, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:36.860655 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb40126-3974-4290-bd91-cafec8cbaa14 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:36.979026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d3867-ba7b-4286-8863-959be60cd5f7 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:36.979316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d3867-ba7b-4286-8863-959be60cd5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:36.979623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d3867-ba7b-4286-8863-959be60cd5f7 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:36.983379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 926/1848] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:36 2022] PUT /identity/v3/projects/1b896eb810a04025b17fff33697fefe8/users/05f29ccf27fa4030b0dbc751a8ca9c35/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:36.985357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451b94-8a4a-46ac-a7fa-dad6e81bf8b9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:36.985867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451b94-8a4a-46ac-a7fa-dad6e81bf8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:36.986348 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451b94-8a4a-46ac-a7fa-dad6e81bf8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:36.988349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 923/1849] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:44:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:44:37.032258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f67ec-069a-478f-8367-5005d784c385 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:37.032530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f67ec-069a-478f-8367-5005d784c385 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:37.032807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f67ec-069a-478f-8367-5005d784c385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:37.099907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-0b4f67ec-069a-478f-8367-5005d784c385 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:37.102307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0b4f67ec-069a-478f-8367-5005d784c385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:37.146568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4c451b94-8a4a-46ac-a7fa-dad6e81bf8b9 None None] MFA Rules not processed for user `05f29ccf27fa4030b0dbc751a8ca9c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:37.152495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4c451b94-8a4a-46ac-a7fa-dad6e81bf8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:37.345525 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 924/1850] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:44:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:37.439198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f67ec-069a-478f-8367-5005d784c385 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:37.441303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0b4f67ec-069a-478f-8367-5005d784c385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:37.476760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 927/1851] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:37.482931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451b94-8a4a-46ac-a7fa-dad6e81bf8b9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:37.485283 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f67ec-069a-478f-8367-5005d784c385 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:37.499954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a0181-ce90-4726-916b-b427f8ead0e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:37.500610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a0181-ce90-4726-916b-b427f8ead0e9 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:37.500878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a0181-ce90-4726-916b-b427f8ead0e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:37.501149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a0181-ce90-4726-916b-b427f8ead0e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:37.522316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a0181-ce90-4726-916b-b427f8ead0e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:37.524219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a0181-ce90-4726-916b-b427f8ead0e9 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:37.563878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 925/1852] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:44:37.678718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451b94-8a4a-46ac-a7fa-dad6e81bf8b9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:37.685209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa627a8-7f25-4bb8-aeb3-2a4fa2b57510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:37.686214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa627a8-7f25-4bb8-aeb3-2a4fa2b57510 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:37.686506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa627a8-7f25-4bb8-aeb3-2a4fa2b57510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:37.686784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa627a8-7f25-4bb8-aeb3-2a4fa2b57510 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:37.687492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa627a8-7f25-4bb8-aeb3-2a4fa2b57510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:37.689466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa627a8-7f25-4bb8-aeb3-2a4fa2b57510 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:37.809711 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 928/1853] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:37 2022] POST /identity/v3/projects => generated 381 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:37.812377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a0181-ce90-4726-916b-b427f8ead0e9 service nova] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:37.827727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a0181-ce90-4726-916b-b427f8ead0e9 service nova] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:37.953077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4efe1-84d2-4373-8440-11e1c2574a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:37.954923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4efe1-84d2-4373-8440-11e1c2574a95 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:37.956990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4efe1-84d2-4373-8440-11e1c2574a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:37.957528 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4efe1-84d2-4373-8440-11e1c2574a95 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:37.958329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4efe1-84d2-4373-8440-11e1c2574a95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:37.960337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4efe1-84d2-4373-8440-11e1c2574a95 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:38.062597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 926/1854] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:37 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:38.068055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa627a8-7f25-4bb8-aeb3-2a4fa2b57510 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:38.089451 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa627a8-7f25-4bb8-aeb3-2a4fa2b57510 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:38.103462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba41424-416a-46b2-bdf8-7e6d7790a3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:38.104641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba41424-416a-46b2-bdf8-7e6d7790a3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:38.105075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba41424-416a-46b2-bdf8-7e6d7790a3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:38.105609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba41424-416a-46b2-bdf8-7e6d7790a3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:38.106541 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba41424-416a-46b2-bdf8-7e6d7790a3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:38.124333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba41424-416a-46b2-bdf8-7e6d7790a3e1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:38.164511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 929/1855] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:38.169245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4efe1-84d2-4373-8440-11e1c2574a95 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:38.185137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4efe1-84d2-4373-8440-11e1c2574a95 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:38.210893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcdaa5-0bf3-49e9-b22e-bb55535f4b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:38.214799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcdaa5-0bf3-49e9-b22e-bb55535f4b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:38.219763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcdaa5-0bf3-49e9-b22e-bb55535f4b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:38.221068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcdaa5-0bf3-49e9-b22e-bb55535f4b5d admin admin] PATH_INFO: `/v3/projects/baee4d32332742b8aa5bc24410eb5e10/users/f9cd78f745a8404fafd0d9cf071d29b3/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:38.271234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcdaa5-0bf3-49e9-b22e-bb55535f4b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=baee4d32332742b8aa5bc24410eb5e10, user_id=f9cd78f745a8404fafd0d9cf071d29b3, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:38.304287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcdaa5-0bf3-49e9-b22e-bb55535f4b5d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:38.411429 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 927/1856] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:38 2022] PUT /identity/v3/projects/baee4d32332742b8aa5bc24410eb5e10/users/f9cd78f745a8404fafd0d9cf071d29b3/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:38.416488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba41424-416a-46b2-bdf8-7e6d7790a3e1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:38.435363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba41424-416a-46b2-bdf8-7e6d7790a3e1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:38.449942 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:38.591157 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a48197-b87d-4886-af8d-87956798a570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:38.591845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a48197-b87d-4886-af8d-87956798a570 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:38.592125 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a48197-b87d-4886-af8d-87956798a570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:38.592399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a48197-b87d-4886-af8d-87956798a570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:38.598356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a48197-b87d-4886-af8d-87956798a570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:38.603233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a48197-b87d-4886-af8d-87956798a570 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:38.621115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 930/1857] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:38.625794 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcdaa5-0bf3-49e9-b22e-bb55535f4b5d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:38.637303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcdaa5-0bf3-49e9-b22e-bb55535f4b5d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:38.642853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e005e-90b0-4f3e-8caf-90560f294efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:38.643764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e005e-90b0-4f3e-8caf-90560f294efc admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:38.644046 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e005e-90b0-4f3e-8caf-90560f294efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:38.644319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e005e-90b0-4f3e-8caf-90560f294efc admin admin] PATH_INFO: `/v3/projects/baee4d32332742b8aa5bc24410eb5e10/users/f9cd78f745a8404fafd0d9cf071d29b3/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:38.646579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e005e-90b0-4f3e-8caf-90560f294efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=baee4d32332742b8aa5bc24410eb5e10, user_id=f9cd78f745a8404fafd0d9cf071d29b3, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:38.650510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e005e-90b0-4f3e-8caf-90560f294efc admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:38.674511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 928/1858] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:38 2022] PUT /identity/v3/projects/baee4d32332742b8aa5bc24410eb5e10/users/f9cd78f745a8404fafd0d9cf071d29b3/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:38.686988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:38.689069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0718f-8d2a-4f13-bd26-9ce4cdec43ca None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:38.689612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0718f-8d2a-4f13-bd26-9ce4cdec43ca None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:38.689688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0718f-8d2a-4f13-bd26-9ce4cdec43ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:38.722903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e5e0718f-8d2a-4f13-bd26-9ce4cdec43ca None None] MFA Rules not processed for user `f9cd78f745a8404fafd0d9cf071d29b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:38.724216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e5e0718f-8d2a-4f13-bd26-9ce4cdec43ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:38.811256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 931/1859] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:38.930886 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e005e-90b0-4f3e-8caf-90560f294efc admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:38.933919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a70e005e-90b0-4f3e-8caf-90560f294efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:38.999933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e005e-90b0-4f3e-8caf-90560f294efc admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:39.009922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ea221-3ad3-4e7e-a360-936be04cca0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:39.010602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ea221-3ad3-4e7e-a360-936be04cca0b service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:39.010890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ea221-3ad3-4e7e-a360-936be04cca0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:39.011167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ea221-3ad3-4e7e-a360-936be04cca0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:39.023353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ea221-3ad3-4e7e-a360-936be04cca0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:39.026268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ea221-3ad3-4e7e-a360-936be04cca0b service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:39.037970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 929/1860] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:39.374440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80866-7064-4f18-a840-fa79a14418cc None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:39.375224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80866-7064-4f18-a840-fa79a14418cc None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:39.375673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80866-7064-4f18-a840-fa79a14418cc None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:39.382498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 932/1861] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:44:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:44:39.391094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94e1a1-8e69-419a-a60b-24a8c821afc7 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:39.393608 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94e1a1-8e69-419a-a60b-24a8c821afc7 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:39.394069 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94e1a1-8e69-419a-a60b-24a8c821afc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:39.423144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-cb94e1a1-8e69-419a-a60b-24a8c821afc7 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:39.424908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cb94e1a1-8e69-419a-a60b-24a8c821afc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:39.522968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 930/1862] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:44:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:39.893047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef80866-7064-4f18-a840-fa79a14418cc None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:40.029482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94e1a1-8e69-419a-a60b-24a8c821afc7 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:40.031478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef80866-7064-4f18-a840-fa79a14418cc None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:40.037464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:40.038500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:40.038852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:40.039218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] PATH_INFO: `/v3/users/05f29ccf27fa4030b0dbc751a8ca9c35` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:40.040584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05f29ccf27fa4030b0dbc751a8ca9c35)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:40.043458 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:40.047689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94e1a1-8e69-419a-a60b-24a8c821afc7 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:40.049953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:40.063041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c138ff9-6500-47f1-a7f3-10f180e02df1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:40.063719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c138ff9-6500-47f1-a7f3-10f180e02df1 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:40.064003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c138ff9-6500-47f1-a7f3-10f180e02df1 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:40.064280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c138ff9-6500-47f1-a7f3-10f180e02df1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:40.076210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c138ff9-6500-47f1-a7f3-10f180e02df1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:40.078432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c138ff9-6500-47f1-a7f3-10f180e02df1 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:40.099092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 931/1863] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:44:40.273036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05f29ccf27fa4030b0dbc751a8ca9c35'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:40.280650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05f29ccf27fa4030b0dbc751a8ca9c35'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:40.293797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05f29ccf27fa4030b0dbc751a8ca9c35'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:40.385191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05f29ccf27fa4030b0dbc751a8ca9c35'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:40.385849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05f29ccf27fa4030b0dbc751a8ca9c35'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:40.392659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 933/1864] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:39 2022] DELETE /identity/v3/users/05f29ccf27fa4030b0dbc751a8ca9c35 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:40.397630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec68cf-a73c-43b6-8a5b-e8d45f54ae00 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:40.398147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec68cf-a73c-43b6-8a5b-e8d45f54ae00 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:40.398807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec68cf-a73c-43b6-8a5b-e8d45f54ae00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:40.417316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c7ec68cf-a73c-43b6-8a5b-e8d45f54ae00 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:40.418520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c7ec68cf-a73c-43b6-8a5b-e8d45f54ae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:40.480440 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 932/1865] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:40.491147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:40.493280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:40.586819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1ae0b-2a14-43bf-b65d-c4c8de0cb6f0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:40.593407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07da7-8c32-499b-9c82-0f17cdebc4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:40.594245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07da7-8c32-499b-9c82-0f17cdebc4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:40.594537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07da7-8c32-499b-9c82-0f17cdebc4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:40.594818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07da7-8c32-499b-9c82-0f17cdebc4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:40.607229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc07da7-8c32-499b-9c82-0f17cdebc4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:40.608696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc07da7-8c32-499b-9c82-0f17cdebc4c9 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:40.621448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 934/1866] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:40.638630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:40.661073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec68cf-a73c-43b6-8a5b-e8d45f54ae00 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:40.662776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c7ec68cf-a73c-43b6-8a5b-e8d45f54ae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:40.676999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec68cf-a73c-43b6-8a5b-e8d45f54ae00 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:40.683145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:40.685013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:40.685644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:40.686109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] PATH_INFO: `/v3/users/1db04445458d4489b8085cbfd58d3c3b` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:40.688767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1db04445458d4489b8085cbfd58d3c3b)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:40.691193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:40.925887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1db04445458d4489b8085cbfd58d3c3b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:40.927189 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1db04445458d4489b8085cbfd58d3c3b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:40.972282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1db04445458d4489b8085cbfd58d3c3b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:40.991874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1db04445458d4489b8085cbfd58d3c3b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:41.074596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1db04445458d4489b8085cbfd58d3c3b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:41.082017 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 933/1867] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:40 2022] DELETE /identity/v3/users/1db04445458d4489b8085cbfd58d3c3b => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:41.086592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e93d7b-be33-4262-9a8c-a530505fa547 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:41.087104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e93d7b-be33-4262-9a8c-a530505fa547 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:41.087203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e93d7b-be33-4262-9a8c-a530505fa547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:41.152952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a6e93d7b-be33-4262-9a8c-a530505fa547 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:41.155803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a6e93d7b-be33-4262-9a8c-a530505fa547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:41.332432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 935/1868] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:41.350533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:41.353069 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:41.551795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac515-d55a-41a2-b079-c4cc9304f963 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:41.563366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f43bed-4c2a-43b2-a057-cd4fd58eccd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:41.564308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f43bed-4c2a-43b2-a057-cd4fd58eccd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:41.569510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f43bed-4c2a-43b2-a057-cd4fd58eccd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:41.569510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f43bed-4c2a-43b2-a057-cd4fd58eccd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:41.582001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f43bed-4c2a-43b2-a057-cd4fd58eccd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:41.583688 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f43bed-4c2a-43b2-a057-cd4fd58eccd9 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:41.594777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 934/1869] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:41.740965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:41.882771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e93d7b-be33-4262-9a8c-a530505fa547 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:41.884393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a6e93d7b-be33-4262-9a8c-a530505fa547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:41.896950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e93d7b-be33-4262-9a8c-a530505fa547 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:41.903657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:41.904745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:41.905035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:41.905363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] PATH_INFO: `/v3/users/69d2b241046f4c7fbedf5c765d92f16d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:41.907140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69d2b241046f4c7fbedf5c765d92f16d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:41.909308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:42.113001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69d2b241046f4c7fbedf5c765d92f16d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.122754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69d2b241046f4c7fbedf5c765d92f16d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.136922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69d2b241046f4c7fbedf5c765d92f16d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.149840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f43bed-4c2a-43b2-a057-cd4fd58eccd9 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:42.152002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d9f43bed-4c2a-43b2-a057-cd4fd58eccd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:42.182623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f43bed-4c2a-43b2-a057-cd4fd58eccd9 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:42.191039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:42.192189 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:42.192573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:42.192928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] PATH_INFO: `/v3/projects/1b896eb810a04025b17fff33697fefe8` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:42.194640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b896eb810a04025b17fff33697fefe8)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:42.196691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:42.250033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69d2b241046f4c7fbedf5c765d92f16d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.251914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-25044327-99ac-4cc0-9ae6-f22c775c4ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69d2b241046f4c7fbedf5c765d92f16d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.261597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 936/1870] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:41 2022] DELETE /identity/v3/users/69d2b241046f4c7fbedf5c765d92f16d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:42.265832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecfe4a-83c7-4f1a-9055-bb885a8f6e2d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:42.266191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecfe4a-83c7-4f1a-9055-bb885a8f6e2d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:42.266543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecfe4a-83c7-4f1a-9055-bb885a8f6e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:42.286934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-16ecfe4a-83c7-4f1a-9055-bb885a8f6e2d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:42.288351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-16ecfe4a-83c7-4f1a-9055-bb885a8f6e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:42.358228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b896eb810a04025b17fff33697fefe8', 'request_id': 'req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.360631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:42.364833 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b896eb810a04025b17fff33697fefe8', 'request_id': 'req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.364969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 937/1871] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:42.375406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b896eb810a04025b17fff33697fefe8', 'request_id': 'req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.378853 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecfe4a-83c7-4f1a-9055-bb885a8f6e2d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:42.380368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-16ecfe4a-83c7-4f1a-9055-bb885a8f6e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:42.441747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecfe4a-83c7-4f1a-9055-bb885a8f6e2d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:42.533794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b9b26-e038-4c87-a632-fc35db0833a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:42.538246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b9b26-e038-4c87-a632-fc35db0833a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:42.538549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b9b26-e038-4c87-a632-fc35db0833a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:42.538819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b9b26-e038-4c87-a632-fc35db0833a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:42.580217 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] The token cache is being invalidate because project 1b896eb810a04025b17fff33697fefe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:42.580511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.586253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.586253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 935/1872] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:42 2022] DELETE /identity/v3/projects/1b896eb810a04025b17fff33697fefe8 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:42.586253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:42.586253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:42.658017 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219a41f-ce78-4218-9d2a-f5ef912d0bf9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:42.663657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:42.664173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b9b26-e038-4c87-a632-fc35db0833a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:42.664685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:42.665068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:42.665482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] PATH_INFO: `/v3/projects/63659efb2a904d348ec7795a11d31edc` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:42.665670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b9b26-e038-4c87-a632-fc35db0833a0 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:42.668139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63659efb2a904d348ec7795a11d31edc)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:42.677548 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 938/1873] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:44:42.678160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b9b26-e038-4c87-a632-fc35db0833a0 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:42.680698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9c9b9b26-e038-4c87-a632-fc35db0833a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:42.682679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:42.700206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b9b26-e038-4c87-a632-fc35db0833a0 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:42.710673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:42.711583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:42.711870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:42.712324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] PATH_INFO: `/v3/users/f9cd78f745a8404fafd0d9cf071d29b3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:42.716892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9cd78f745a8404fafd0d9cf071d29b3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:42.723991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:42.799329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63659efb2a904d348ec7795a11d31edc', 'request_id': 'req-7fe00109-2273-42e3-91db-50440f090362'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.806026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63659efb2a904d348ec7795a11d31edc', 'request_id': 'req-7fe00109-2273-42e3-91db-50440f090362'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.817514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63659efb2a904d348ec7795a11d31edc', 'request_id': 'req-7fe00109-2273-42e3-91db-50440f090362'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.924232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] The token cache is being invalidate because project 63659efb2a904d348ec7795a11d31edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:42.924514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.925151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:42.926749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 936/1874] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:42 2022] DELETE /identity/v3/projects/63659efb2a904d348ec7795a11d31edc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:42.930752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:42.932683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:43.001303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00109-2273-42e3-91db-50440f090362 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:43.006583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:43.007941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:43.007941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:43.008280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] PATH_INFO: `/v3/users/5310a53319c94ab49a5a5d757bdc9ad1` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:43.009627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5310a53319c94ab49a5a5d757bdc9ad1)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:43.011709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:43.049146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9cd78f745a8404fafd0d9cf071d29b3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.056382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9cd78f745a8404fafd0d9cf071d29b3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.070895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9cd78f745a8404fafd0d9cf071d29b3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.151267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9cd78f745a8404fafd0d9cf071d29b3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.152273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9cd78f745a8404fafd0d9cf071d29b3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.161706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 939/1875] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:42 2022] DELETE /identity/v3/users/f9cd78f745a8404fafd0d9cf071d29b3 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:43.257877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5310a53319c94ab49a5a5d757bdc9ad1'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.258647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5310a53319c94ab49a5a5d757bdc9ad1'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.271153 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5310a53319c94ab49a5a5d757bdc9ad1'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.279045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5310a53319c94ab49a5a5d757bdc9ad1'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.381580 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:43.383342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:43.427286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5310a53319c94ab49a5a5d757bdc9ad1'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.435887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 937/1876] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:42 2022] DELETE /identity/v3/users/5310a53319c94ab49a5a5d757bdc9ad1 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:43.492502 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d6ed5-89c1-4ffa-af7d-8ac2cbb13880 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:43.605576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:43.606440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:43.606822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:43.607039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] PATH_INFO: `/v3/projects/7ec2443e342f45bbb5140cc2168dab1e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:43.608648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec2443e342f45bbb5140cc2168dab1e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:43.610615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:43.788583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec2443e342f45bbb5140cc2168dab1e', 'request_id': 'req-1207181f-fb83-454c-828a-7de42a473509'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.819653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:43.821468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:43.843386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a05a0b-cda8-4c93-bfd1-fb78e2107ef3 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:43.917653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec2443e342f45bbb5140cc2168dab1e', 'request_id': 'req-1207181f-fb83-454c-828a-7de42a473509'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.926389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec2443e342f45bbb5140cc2168dab1e', 'request_id': 'req-1207181f-fb83-454c-828a-7de42a473509'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.949607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] The token cache is being invalidate because project 7ec2443e342f45bbb5140cc2168dab1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:43.950132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.951281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:43.954244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 940/1877] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:43 2022] DELETE /identity/v3/projects/7ec2443e342f45bbb5140cc2168dab1e => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:43.958402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:43.964256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:43.966881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:43.967917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:43.968205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:43.968478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] PATH_INFO: `/v3/projects/baee4d32332742b8aa5bc24410eb5e10` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:43.969889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baee4d32332742b8aa5bc24410eb5e10)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:43.972844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:43.999658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207181f-fb83-454c-828a-7de42a473509 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:44.022958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:44.024290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.024679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.025052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] PATH_INFO: `/v3/users/8d2e4db18d974872816b84cc616a2de8` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.027191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d2e4db18d974872816b84cc616a2de8)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:44.029273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:44.262935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d2e4db18d974872816b84cc616a2de8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.266009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baee4d32332742b8aa5bc24410eb5e10', 'request_id': 'req-cfb4aab8-147e-4991-bf25-19e75fd89cb0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.271752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d2e4db18d974872816b84cc616a2de8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.279463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baee4d32332742b8aa5bc24410eb5e10', 'request_id': 'req-cfb4aab8-147e-4991-bf25-19e75fd89cb0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.285259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d2e4db18d974872816b84cc616a2de8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.292148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baee4d32332742b8aa5bc24410eb5e10', 'request_id': 'req-cfb4aab8-147e-4991-bf25-19e75fd89cb0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.358969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d2e4db18d974872816b84cc616a2de8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.360890 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d2e4db18d974872816b84cc616a2de8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.371619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 941/1878] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:43 2022] DELETE /identity/v3/users/8d2e4db18d974872816b84cc616a2de8 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:44.371619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:44.379688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:44.431288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] The token cache is being invalidate because project baee4d32332742b8aa5bc24410eb5e10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:44.431650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.434553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-cfb4aab8-147e-4991-bf25-19e75fd89cb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.437029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 938/1879] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:43 2022] DELETE /identity/v3/projects/baee4d32332742b8aa5bc24410eb5e10 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:44.441291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aef9a-2852-4536-844e-6a1a6d7de491 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.449810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aef9a-2852-4536-844e-6a1a6d7de491 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.449810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aef9a-2852-4536-844e-6a1a6d7de491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.466076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-493aef9a-2852-4536-844e-6a1a6d7de491 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:44.468292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-493aef9a-2852-4536-844e-6a1a6d7de491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:44.478065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf12878-ba76-4299-a563-a2a35b8d0215 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:44.489194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 939/1880] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:44.489366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:44.489730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.489957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.495687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] PATH_INFO: `/v3/projects/8df00e04f89445e887672557af2551fc` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.495687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df00e04f89445e887672557af2551fc)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:44.495687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:44.495960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255417f4-f2d9-4068-abe6-edfb5860b08f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.495960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255417f4-f2d9-4068-abe6-edfb5860b08f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.495960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255417f4-f2d9-4068-abe6-edfb5860b08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.518335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-255417f4-f2d9-4068-abe6-edfb5860b08f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:44.519622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-255417f4-f2d9-4068-abe6-edfb5860b08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:44.537476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 940/1881] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:44.538648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255417f4-f2d9-4068-abe6-edfb5860b08f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:44.550031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255417f4-f2d9-4068-abe6-edfb5860b08f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:44.555592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c193-e517-4ea2-ae29-39b44b9e285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:44.556368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c193-e517-4ea2-ae29-39b44b9e285f admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.556699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c193-e517-4ea2-ae29-39b44b9e285f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.556999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c193-e517-4ea2-ae29-39b44b9e285f admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.557828 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572c193-e517-4ea2-ae29-39b44b9e285f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:44.558207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572c193-e517-4ea2-ae29-39b44b9e285f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:44.559874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572c193-e517-4ea2-ae29-39b44b9e285f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:44.574348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 941/1882] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:44:44.578944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c193-e517-4ea2-ae29-39b44b9e285f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:44.589732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c193-e517-4ea2-ae29-39b44b9e285f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:44.595321 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147caa5b-b71c-4a4c-9e97-9ec502819720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:44.596016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147caa5b-b71c-4a4c-9e97-9ec502819720 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.596301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147caa5b-b71c-4a4c-9e97-9ec502819720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.596789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147caa5b-b71c-4a4c-9e97-9ec502819720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.597547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147caa5b-b71c-4a4c-9e97-9ec502819720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:44.599489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147caa5b-b71c-4a4c-9e97-9ec502819720 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:44.621802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df00e04f89445e887672557af2551fc', 'request_id': 'req-8ecd298a-b593-417a-a3ba-fc8480d37c2a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.691396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 942/1883] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:44 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:44:44.695419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147caa5b-b71c-4a4c-9e97-9ec502819720 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:44.720606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df00e04f89445e887672557af2551fc', 'request_id': 'req-8ecd298a-b593-417a-a3ba-fc8480d37c2a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.729922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df00e04f89445e887672557af2551fc', 'request_id': 'req-8ecd298a-b593-417a-a3ba-fc8480d37c2a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.742794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] The token cache is being invalidate because project 8df00e04f89445e887672557af2551fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:44.743319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.744572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8ecd298a-b593-417a-a3ba-fc8480d37c2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:44.746709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 942/1884] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:44 2022] DELETE /identity/v3/projects/8df00e04f89445e887672557af2551fc => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:44.751109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b26fa1-bd08-4f79-844b-45aa95b97016 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.751404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b26fa1-bd08-4f79-844b-45aa95b97016 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.751691 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b26fa1-bd08-4f79-844b-45aa95b97016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.771783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-83b26fa1-bd08-4f79-844b-45aa95b97016 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:44.773005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-83b26fa1-bd08-4f79-844b-45aa95b97016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:44.786432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147caa5b-b71c-4a4c-9e97-9ec502819720 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:44.792009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873354c2-2cfd-440b-8e7f-5eab2fa49ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:44.792875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873354c2-2cfd-440b-8e7f-5eab2fa49ebf admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.793291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873354c2-2cfd-440b-8e7f-5eab2fa49ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.793652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873354c2-2cfd-440b-8e7f-5eab2fa49ebf admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.794427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873354c2-2cfd-440b-8e7f-5eab2fa49ebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:44.796366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873354c2-2cfd-440b-8e7f-5eab2fa49ebf admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:44.842007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 943/1885] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:44.850639 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24dba-f5e1-4f06-a6e6-4530046051d7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.850933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24dba-f5e1-4f06-a6e6-4530046051d7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.851208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24dba-f5e1-4f06-a6e6-4530046051d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.869944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d8f24dba-f5e1-4f06-a6e6-4530046051d7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:44.871159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d8f24dba-f5e1-4f06-a6e6-4530046051d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:44.879158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 944/1886] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:44.883203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f24dba-f5e1-4f06-a6e6-4530046051d7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:44.892818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f24dba-f5e1-4f06-a6e6-4530046051d7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:44.897703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c7827-7fff-49ec-b8d2-6d39ee1177f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:44.898369 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c7827-7fff-49ec-b8d2-6d39ee1177f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.898953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c7827-7fff-49ec-b8d2-6d39ee1177f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.899198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c7827-7fff-49ec-b8d2-6d39ee1177f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.899935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c7827-7fff-49ec-b8d2-6d39ee1177f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:44.900276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c7827-7fff-49ec-b8d2-6d39ee1177f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:44.901927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c7827-7fff-49ec-b8d2-6d39ee1177f2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:44.919206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 945/1887] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:44:44.920099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c7827-7fff-49ec-b8d2-6d39ee1177f2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:44.931273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c7827-7fff-49ec-b8d2-6d39ee1177f2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:44.937569 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751780b-4579-4eb3-8bae-22a53229ce57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:44.938412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751780b-4579-4eb3-8bae-22a53229ce57 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:44.938848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751780b-4579-4eb3-8bae-22a53229ce57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:44.939658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751780b-4579-4eb3-8bae-22a53229ce57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:44.940632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751780b-4579-4eb3-8bae-22a53229ce57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:44.944236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751780b-4579-4eb3-8bae-22a53229ce57 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:44.977137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 943/1888] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:44 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:44.977137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873354c2-2cfd-440b-8e7f-5eab2fa49ebf admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:44.977137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-873354c2-2cfd-440b-8e7f-5eab2fa49ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:44.990493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873354c2-2cfd-440b-8e7f-5eab2fa49ebf admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:44.999279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7659c8-beef-4fbe-87d7-29be2b85978d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.000021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7659c8-beef-4fbe-87d7-29be2b85978d admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.000358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7659c8-beef-4fbe-87d7-29be2b85978d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.000701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7659c8-beef-4fbe-87d7-29be2b85978d admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.010930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7659c8-beef-4fbe-87d7-29be2b85978d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.010930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:45.010930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7659c8-beef-4fbe-87d7-29be2b85978d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.020657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 944/1889] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:44:45.020986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7659c8-beef-4fbe-87d7-29be2b85978d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:45.023896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-dc7659c8-beef-4fbe-87d7-29be2b85978d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:45.041661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7659c8-beef-4fbe-87d7-29be2b85978d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:45.048345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.049241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.049532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.049831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] PATH_INFO: `/v3/projects/2f87c9c6faef479fa7febf9d21c7fa0f` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.051073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f87c9c6faef479fa7febf9d21c7fa0f)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.053026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.121659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 946/1890] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:44 2022] POST /identity/v3/projects => generated 359 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:45.122501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751780b-4579-4eb3-8bae-22a53229ce57 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:45.200663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751780b-4579-4eb3-8bae-22a53229ce57 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:45.200853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f87c9c6faef479fa7febf9d21c7fa0f', 'request_id': 'req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:45.208522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f87c9c6faef479fa7febf9d21c7fa0f', 'request_id': 'req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:45.216903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f87c9c6faef479fa7febf9d21c7fa0f', 'request_id': 'req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:45.276539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff60a-9f5d-4f23-8ad2-70bf66d5efa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.277696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff60a-9f5d-4f23-8ad2-70bf66d5efa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.277696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff60a-9f5d-4f23-8ad2-70bf66d5efa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.277939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff60a-9f5d-4f23-8ad2-70bf66d5efa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.278869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff60a-9f5d-4f23-8ad2-70bf66d5efa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.280465 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff60a-9f5d-4f23-8ad2-70bf66d5efa8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.295281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 947/1891] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:45.295281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff60a-9f5d-4f23-8ad2-70bf66d5efa8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:45.298560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] The token cache is being invalidate because project 2f87c9c6faef479fa7febf9d21c7fa0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:45.302622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:45.302622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:45.302622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 945/1892] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:45 2022] DELETE /identity/v3/projects/2f87c9c6faef479fa7febf9d21c7fa0f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:45.305531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:45.307976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff60a-9f5d-4f23-8ad2-70bf66d5efa8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:45.308135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:45.313957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad52298-04f6-4313-9b4b-bcad80e28823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.314760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad52298-04f6-4313-9b4b-bcad80e28823 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.315282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad52298-04f6-4313-9b4b-bcad80e28823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.315738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad52298-04f6-4313-9b4b-bcad80e28823 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.316571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad52298-04f6-4313-9b4b-bcad80e28823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.318723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad52298-04f6-4313-9b4b-bcad80e28823 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.321781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c503ef3-d5e8-4f2c-acf2-fb7c2c780dce admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:45.327350 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac5f7b-d0e1-4438-a911-acd9a4877a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.328268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac5f7b-d0e1-4438-a911-acd9a4877a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.328556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac5f7b-d0e1-4438-a911-acd9a4877a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.328838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac5f7b-d0e1-4438-a911-acd9a4877a99 admin admin] PATH_INFO: `/v3/projects/c28cb6b831b4427e8d84ac316363c73c/users/925240e79ae64724b059e76190471059/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.344636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac5f7b-d0e1-4438-a911-acd9a4877a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28cb6b831b4427e8d84ac316363c73c, user_id=925240e79ae64724b059e76190471059, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.347340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac5f7b-d0e1-4438-a911-acd9a4877a99 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.441128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 948/1893] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:45 2022] POST /identity/v3/users => generated 280 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:45.442753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b351a07-4b11-4eb5-ac33-a18f2a1da42d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.443285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b351a07-4b11-4eb5-ac33-a18f2a1da42d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.444201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b351a07-4b11-4eb5-ac33-a18f2a1da42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.444758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 946/1894] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:45 2022] PUT /identity/v3/projects/c28cb6b831b4427e8d84ac316363c73c/users/925240e79ae64724b059e76190471059/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:45.445873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac5f7b-d0e1-4438-a911-acd9a4877a99 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:45.455194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4cac5f7b-d0e1-4438-a911-acd9a4877a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:45.465642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9b351a07-4b11-4eb5-ac33-a18f2a1da42d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:45.466856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9b351a07-4b11-4eb5-ac33-a18f2a1da42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:45.544507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 949/1895] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:45.544507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ebc2e-60af-4930-bc92-a30c2ee6573a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.544507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ebc2e-60af-4930-bc92-a30c2ee6573a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.544507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ebc2e-60af-4930-bc92-a30c2ee6573a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.565118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac5f7b-d0e1-4438-a911-acd9a4877a99 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:45.568066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6b7ebc2e-60af-4930-bc92-a30c2ee6573a None None] MFA Rules not processed for user `925240e79ae64724b059e76190471059`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:45.569398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6b7ebc2e-60af-4930-bc92-a30c2ee6573a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:45.571023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f37952-174d-4e9d-a97f-45bbd22ba215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.571809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f37952-174d-4e9d-a97f-45bbd22ba215 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.572331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f37952-174d-4e9d-a97f-45bbd22ba215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.576767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f37952-174d-4e9d-a97f-45bbd22ba215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.576767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f37952-174d-4e9d-a97f-45bbd22ba215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.576767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f37952-174d-4e9d-a97f-45bbd22ba215 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.587669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 947/1896] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:45.589820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903661fc-2207-445a-b26c-370a6ebd351e None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.590261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903661fc-2207-445a-b26c-370a6ebd351e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.590544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903661fc-2207-445a-b26c-370a6ebd351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.613479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-903661fc-2207-445a-b26c-370a6ebd351e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:45.614860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-903661fc-2207-445a-b26c-370a6ebd351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:45.625044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 948/1897] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:45.626023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903661fc-2207-445a-b26c-370a6ebd351e None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:45.633993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:45.641512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903661fc-2207-445a-b26c-370a6ebd351e None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:45.647791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f6985-060c-435e-be40-8250d3f5e2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.649547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f6985-060c-435e-be40-8250d3f5e2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.649834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f6985-060c-435e-be40-8250d3f5e2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.650113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f6985-060c-435e-be40-8250d3f5e2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.651398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f6985-060c-435e-be40-8250d3f5e2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.653113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f6985-060c-435e-be40-8250d3f5e2a2 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.683023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 949/1898] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:45.683262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f6985-060c-435e-be40-8250d3f5e2a2 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:45.692036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 950/1899] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:45.692923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ebc2e-60af-4930-bc92-a30c2ee6573a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:45.717212 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f6985-060c-435e-be40-8250d3f5e2a2 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:45.734280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11af41-2d97-473d-9bbe-8c94ddc0d6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.734830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11af41-2d97-473d-9bbe-8c94ddc0d6da admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.735255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11af41-2d97-473d-9bbe-8c94ddc0d6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.735561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11af41-2d97-473d-9bbe-8c94ddc0d6da admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.736461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11af41-2d97-473d-9bbe-8c94ddc0d6da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:45.736866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11af41-2d97-473d-9bbe-8c94ddc0d6da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.738666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11af41-2d97-473d-9bbe-8c94ddc0d6da admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.744808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ebc2e-60af-4930-bc92-a30c2ee6573a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:45.755385 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e24c9-2a11-4602-bb57-3e38bf1d902f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.757273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e24c9-2a11-4602-bb57-3e38bf1d902f admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.766339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e24c9-2a11-4602-bb57-3e38bf1d902f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.766339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e24c9-2a11-4602-bb57-3e38bf1d902f admin admin] PATH_INFO: `/v3/projects/72848b1b00974677ad32505a5f2087bf/users/bec46d9115db4be796a689a5c43b4f1c/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.766698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 950/1900] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:45.767141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11af41-2d97-473d-9bbe-8c94ddc0d6da admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:45.770345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4d11af41-2d97-473d-9bbe-8c94ddc0d6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:45.781747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e24c9-2a11-4602-bb57-3e38bf1d902f admin admin] RBAC: Authorizing `identity:create_grant(project_id=72848b1b00974677ad32505a5f2087bf, user_id=bec46d9115db4be796a689a5c43b4f1c, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.784446 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e24c9-2a11-4602-bb57-3e38bf1d902f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.850256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11af41-2d97-473d-9bbe-8c94ddc0d6da admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:45.855168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d7b05-433b-402d-91a4-a058c2668b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.856052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d7b05-433b-402d-91a4-a058c2668b6b service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.856588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d7b05-433b-402d-91a4-a058c2668b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.857078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d7b05-433b-402d-91a4-a058c2668b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.868090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d7b05-433b-402d-91a4-a058c2668b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.870717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d7b05-433b-402d-91a4-a058c2668b6b service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.883784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 951/1901] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:45.885260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d7b05-433b-402d-91a4-a058c2668b6b service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:45.897671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d7b05-433b-402d-91a4-a058c2668b6b service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:45.905938 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304586b-e16b-4772-b2f1-1ff661dfee71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:45.906682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304586b-e16b-4772-b2f1-1ff661dfee71 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:45.907410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304586b-e16b-4772-b2f1-1ff661dfee71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:45.907701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304586b-e16b-4772-b2f1-1ff661dfee71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:45.908495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304586b-e16b-4772-b2f1-1ff661dfee71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:45.922530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304586b-e16b-4772-b2f1-1ff661dfee71 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:45.931603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 951/1902] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:45 2022] PUT /identity/v3/projects/72848b1b00974677ad32505a5f2087bf/users/bec46d9115db4be796a689a5c43b4f1c/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:45.932940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e24c9-2a11-4602-bb57-3e38bf1d902f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:46.012325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e24c9-2a11-4602-bb57-3e38bf1d902f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:46.017278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:46.018936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:46.019318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:46.019671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] PATH_INFO: `/v3/users/925240e79ae64724b059e76190471059` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:46.021080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=925240e79ae64724b059e76190471059)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:46.023178 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:46.042783 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 952/1903] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:45 2022] POST /identity/v3/projects => generated 369 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:46.047368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fc85b-56c3-4a64-a852-1d7b5a9a3abd None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:46.047872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fc85b-56c3-4a64-a852-1d7b5a9a3abd None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:46.048311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fc85b-56c3-4a64-a852-1d7b5a9a3abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:46.079181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-358fc85b-56c3-4a64-a852-1d7b5a9a3abd None None] MFA Rules not processed for user `bec46d9115db4be796a689a5c43b4f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:46.081045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-358fc85b-56c3-4a64-a852-1d7b5a9a3abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:46.154171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 953/1904] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:46 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:46.155012 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fc85b-56c3-4a64-a852-1d7b5a9a3abd None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:46.257639 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fc85b-56c3-4a64-a852-1d7b5a9a3abd None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:46.263865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441356a-337f-4605-8bd2-5abaa52d351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:46.265735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441356a-337f-4605-8bd2-5abaa52d351e admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:46.265735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441356a-337f-4605-8bd2-5abaa52d351e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:46.265735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441356a-337f-4605-8bd2-5abaa52d351e admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:46.266303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441356a-337f-4605-8bd2-5abaa52d351e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:46.268538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441356a-337f-4605-8bd2-5abaa52d351e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:46.277969 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '925240e79ae64724b059e76190471059'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:46.285670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '925240e79ae64724b059e76190471059'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:46.298656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '925240e79ae64724b059e76190471059'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:46.380404 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 954/1905] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:46 2022] POST /identity/v3/users => generated 285 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:46.382306 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441356a-337f-4605-8bd2-5abaa52d351e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:46.445005 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441356a-337f-4605-8bd2-5abaa52d351e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:46.449211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af7786-4392-4e86-b9bc-12d04c040b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:46.449865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af7786-4392-4e86-b9bc-12d04c040b71 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:46.450147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af7786-4392-4e86-b9bc-12d04c040b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:46.450421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af7786-4392-4e86-b9bc-12d04c040b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:46.459097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af7786-4392-4e86-b9bc-12d04c040b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:46.460539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af7786-4392-4e86-b9bc-12d04c040b71 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:46.470240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 955/1906] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:46 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:46.472752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af7786-4392-4e86-b9bc-12d04c040b71 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:46.489100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af7786-4392-4e86-b9bc-12d04c040b71 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:46.495073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279b277-fbc0-48f4-8181-5d09a2f28970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:46.495918 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279b277-fbc0-48f4-8181-5d09a2f28970 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:46.496209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279b277-fbc0-48f4-8181-5d09a2f28970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:46.496488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279b277-fbc0-48f4-8181-5d09a2f28970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:46.497271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279b277-fbc0-48f4-8181-5d09a2f28970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:46.498945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279b277-fbc0-48f4-8181-5d09a2f28970 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:46.512629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 956/1907] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:46.516575 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279b277-fbc0-48f4-8181-5d09a2f28970 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:46.528396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279b277-fbc0-48f4-8181-5d09a2f28970 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:46.539702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:46.539702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:46.539702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:46.539702 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] PATH_INFO: `/v3/users/bec46d9115db4be796a689a5c43b4f1c` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:46.540397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec46d9115db4be796a689a5c43b4f1c)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:46.540397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:46.599869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '925240e79ae64724b059e76190471059'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:46.601563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '925240e79ae64724b059e76190471059'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:46.610662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 952/1908] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:45 2022] DELETE /identity/v3/users/925240e79ae64724b059e76190471059 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:46.611737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:46.613061 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:46.698462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a436d-d36b-4231-9640-113abd5bef77 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:46.705631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf8cef-a4ee-4f50-8adf-fbfbe283340f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:46.706323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf8cef-a4ee-4f50-8adf-fbfbe283340f admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:46.706614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf8cef-a4ee-4f50-8adf-fbfbe283340f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:46.706915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf8cef-a4ee-4f50-8adf-fbfbe283340f admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:46.707830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf8cef-a4ee-4f50-8adf-fbfbe283340f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:46.709476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf8cef-a4ee-4f50-8adf-fbfbe283340f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:46.722517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 953/1909] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:46.724402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c00af4-cd32-4a58-a251-489dac19d3b3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:46.729530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c00af4-cd32-4a58-a251-489dac19d3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:46.729530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c00af4-cd32-4a58-a251-489dac19d3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:46.762989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-90c00af4-cd32-4a58-a251-489dac19d3b3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:46.764918 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-90c00af4-cd32-4a58-a251-489dac19d3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:46.778807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 954/1910] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:46.778807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c00af4-cd32-4a58-a251-489dac19d3b3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:46.788921 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c00af4-cd32-4a58-a251-489dac19d3b3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:46.794653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda0843-8f8f-4299-bb04-7dc4cc317c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:46.795644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0843-8f8f-4299-bb04-7dc4cc317c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:46.795922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0843-8f8f-4299-bb04-7dc4cc317c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:46.796254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0843-8f8f-4299-bb04-7dc4cc317c4c admin admin] PATH_INFO: `/v3/projects/5995536198644c6099cb9a5ebf8f7b27/users/4c371f62facd40bc96e98a3e0e61e2f6/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:46.814320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda0843-8f8f-4299-bb04-7dc4cc317c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5995536198644c6099cb9a5ebf8f7b27, user_id=4c371f62facd40bc96e98a3e0e61e2f6, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:46.817108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda0843-8f8f-4299-bb04-7dc4cc317c4c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:46.842013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec46d9115db4be796a689a5c43b4f1c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:46.842989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec46d9115db4be796a689a5c43b4f1c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:46.858954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec46d9115db4be796a689a5c43b4f1c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:46.884146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec46d9115db4be796a689a5c43b4f1c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:46.955277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 955/1911] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:46 2022] PUT /identity/v3/projects/5995536198644c6099cb9a5ebf8f7b27/users/4c371f62facd40bc96e98a3e0e61e2f6/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:46.956564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda0843-8f8f-4299-bb04-7dc4cc317c4c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:46.958140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fcda0843-8f8f-4299-bb04-7dc4cc317c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:46.980874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:46.981684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-847ac006-15dc-4333-a804-d0927c6393ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec46d9115db4be796a689a5c43b4f1c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:46.989233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 957/1912] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:46 2022] DELETE /identity/v3/users/bec46d9115db4be796a689a5c43b4f1c => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:46.992556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e3ed-c20b-4414-b99c-71dfaeace9e6 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:46.992556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e3ed-c20b-4414-b99c-71dfaeace9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:46.992556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e3ed-c20b-4414-b99c-71dfaeace9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:47.040838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-f321e3ed-c20b-4414-b99c-71dfaeace9e6 None None] MFA Rules not processed for user `4c371f62facd40bc96e98a3e0e61e2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:47.047959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f321e3ed-c20b-4414-b99c-71dfaeace9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:47.073439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda0843-8f8f-4299-bb04-7dc4cc317c4c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:47.142611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5406b-e5f2-4806-abe5-a37558b2502e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:47.143595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5406b-e5f2-4806-abe5-a37558b2502e service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:47.143889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5406b-e5f2-4806-abe5-a37558b2502e service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:47.144161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5406b-e5f2-4806-abe5-a37558b2502e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:47.148529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 958/1913] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:47.148970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-31c5406b-e5f2-4806-abe5-a37558b2502e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:47.149078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec10b9-8415-4608-8f4d-8b43ab7fb0dc None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:47.149078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec10b9-8415-4608-8f4d-8b43ab7fb0dc None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:47.149078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec10b9-8415-4608-8f4d-8b43ab7fb0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:47.174300 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-36ec10b9-8415-4608-8f4d-8b43ab7fb0dc None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:47.175797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-36ec10b9-8415-4608-8f4d-8b43ab7fb0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:47.235787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5406b-e5f2-4806-abe5-a37558b2502e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:47.237890 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5406b-e5f2-4806-abe5-a37558b2502e service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:47.249460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 956/1914] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:47.250416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5406b-e5f2-4806-abe5-a37558b2502e service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:47.252081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-31c5406b-e5f2-4806-abe5-a37558b2502e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:47.257564 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 959/1915] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:47.266011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5406b-e5f2-4806-abe5-a37558b2502e service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:47.271331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba1912-713d-445c-98c9-8e7c71e2bdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:47.272050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba1912-713d-445c-98c9-8e7c71e2bdd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:47.273103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba1912-713d-445c-98c9-8e7c71e2bdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:47.273444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba1912-713d-445c-98c9-8e7c71e2bdd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:47.274483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba1912-713d-445c-98c9-8e7c71e2bdd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:47.277062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba1912-713d-445c-98c9-8e7c71e2bdd9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:47.279089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec10b9-8415-4608-8f4d-8b43ab7fb0dc None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:47.300761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec10b9-8415-4608-8f4d-8b43ab7fb0dc None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:47.308444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357856-a544-4c23-9230-7b3f119364d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:47.309901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357856-a544-4c23-9230-7b3f119364d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:47.309901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357856-a544-4c23-9230-7b3f119364d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:47.309901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357856-a544-4c23-9230-7b3f119364d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:47.321919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94357856-a544-4c23-9230-7b3f119364d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:47.324681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94357856-a544-4c23-9230-7b3f119364d3 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:47.350767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 960/1916] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:47.437540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 957/1917] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:47 2022] POST /identity/v3/projects => generated 367 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:47.439088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357856-a544-4c23-9230-7b3f119364d3 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:47.454019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357856-a544-4c23-9230-7b3f119364d3 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:47.483404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:47.554982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a410ee4-93e3-4d03-b448-d8c232e645dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:47.555862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a410ee4-93e3-4d03-b448-d8c232e645dd admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:47.556282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a410ee4-93e3-4d03-b448-d8c232e645dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:47.556691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a410ee4-93e3-4d03-b448-d8c232e645dd admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:47.557582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a410ee4-93e3-4d03-b448-d8c232e645dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:47.560530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a410ee4-93e3-4d03-b448-d8c232e645dd admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:47.687632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 961/1918] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:47 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:44:47.689685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba1912-713d-445c-98c9-8e7c71e2bdd9 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:47.716749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba1912-713d-445c-98c9-8e7c71e2bdd9 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:47.726525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb195b4-c068-4401-a67c-f8c89d3ed3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:47.727344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb195b4-c068-4401-a67c-f8c89d3ed3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:47.727753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb195b4-c068-4401-a67c-f8c89d3ed3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:47.728129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb195b4-c068-4401-a67c-f8c89d3ed3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:47.729011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb195b4-c068-4401-a67c-f8c89d3ed3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:47.730742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb195b4-c068-4401-a67c-f8c89d3ed3a5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:47.745915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 958/1919] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:47.753247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a410ee4-93e3-4d03-b448-d8c232e645dd admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:47.767294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a410ee4-93e3-4d03-b448-d8c232e645dd admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:47.774362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea54f59-78b3-40db-be6b-226c83227cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:47.776419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea54f59-78b3-40db-be6b-226c83227cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:47.776784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea54f59-78b3-40db-be6b-226c83227cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:47.777216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea54f59-78b3-40db-be6b-226c83227cbc admin admin] PATH_INFO: `/v3/projects/1506bdca26c14b14953c304c005bace5/users/4dc2b3e9b84440f594a462955b7aed40/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:47.793893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea54f59-78b3-40db-be6b-226c83227cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1506bdca26c14b14953c304c005bace5, user_id=4dc2b3e9b84440f594a462955b7aed40, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:47.801582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea54f59-78b3-40db-be6b-226c83227cbc admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:47.817378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb195b4-c068-4401-a67c-f8c89d3ed3a5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:47.819190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-beb195b4-c068-4401-a67c-f8c89d3ed3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:47.857473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb195b4-c068-4401-a67c-f8c89d3ed3a5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:47.870566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:47.871630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:47.872019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:47.872523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] PATH_INFO: `/v3/projects/c28cb6b831b4427e8d84ac316363c73c` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:47.875217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c28cb6b831b4427e8d84ac316363c73c)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:47.877477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:47.894043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 962/1920] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:47 2022] PUT /identity/v3/projects/1506bdca26c14b14953c304c005bace5/users/4dc2b3e9b84440f594a462955b7aed40/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:47.905667 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea54f59-78b3-40db-be6b-226c83227cbc admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:48.014507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea54f59-78b3-40db-be6b-226c83227cbc admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:48.014507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991ca76-66ae-4100-aef1-f7e7868c2b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:48.014507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991ca76-66ae-4100-aef1-f7e7868c2b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:48.014507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991ca76-66ae-4100-aef1-f7e7868c2b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:48.014975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991ca76-66ae-4100-aef1-f7e7868c2b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:48.014975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991ca76-66ae-4100-aef1-f7e7868c2b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:48.016994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991ca76-66ae-4100-aef1-f7e7868c2b5a admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:48.038377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 963/1921] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:48.038377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991ca76-66ae-4100-aef1-f7e7868c2b5a admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:48.049076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991ca76-66ae-4100-aef1-f7e7868c2b5a admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:48.052509 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c28cb6b831b4427e8d84ac316363c73c', 'request_id': 'req-5b9acc62-3b11-4825-8715-28b5d46d17a9'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:48.129430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7db9b0-c2e2-4e31-87c9-a5843961381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:48.164546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7db9b0-c2e2-4e31-87c9-a5843961381b admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:48.164546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7db9b0-c2e2-4e31-87c9-a5843961381b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:48.164546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7db9b0-c2e2-4e31-87c9-a5843961381b admin admin] PATH_INFO: `/v3/projects/1506bdca26c14b14953c304c005bace5/users/4dc2b3e9b84440f594a462955b7aed40/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:48.164546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7db9b0-c2e2-4e31-87c9-a5843961381b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1506bdca26c14b14953c304c005bace5, user_id=4dc2b3e9b84440f594a462955b7aed40, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:48.164546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7db9b0-c2e2-4e31-87c9-a5843961381b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:48.165449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 964/1922] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:48 2022] PUT /identity/v3/projects/1506bdca26c14b14953c304c005bace5/users/4dc2b3e9b84440f594a462955b7aed40/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:48.165449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7db9b0-c2e2-4e31-87c9-a5843961381b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:48.169335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3b7db9b0-c2e2-4e31-87c9-a5843961381b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:48.238282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c28cb6b831b4427e8d84ac316363c73c', 'request_id': 'req-5b9acc62-3b11-4825-8715-28b5d46d17a9'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:48.250296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c28cb6b831b4427e8d84ac316363c73c', 'request_id': 'req-5b9acc62-3b11-4825-8715-28b5d46d17a9'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:48.280334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] The token cache is being invalidate because project c28cb6b831b4427e8d84ac316363c73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:48.280633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:48.281548 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5b9acc62-3b11-4825-8715-28b5d46d17a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:48.285068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 959/1923] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:47 2022] DELETE /identity/v3/projects/c28cb6b831b4427e8d84ac316363c73c => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:48.286656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638698e-24f8-44bd-8802-7c371c70aa5a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:48.287098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638698e-24f8-44bd-8802-7c371c70aa5a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:48.287521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638698e-24f8-44bd-8802-7c371c70aa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:48.342754 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7db9b0-c2e2-4e31-87c9-a5843961381b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:48.349205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2638698e-24f8-44bd-8802-7c371c70aa5a None None] MFA Rules not processed for user `4dc2b3e9b84440f594a462955b7aed40`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:48.351284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2638698e-24f8-44bd-8802-7c371c70aa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:48.358001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:48.358900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:48.359183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:48.359572 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] PATH_INFO: `/v3/projects/72848b1b00974677ad32505a5f2087bf` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:48.361486 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72848b1b00974677ad32505a5f2087bf)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:48.363547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:48.428297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:48.462329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 960/1924] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:48 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:48.464084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1492bd-ab3d-48f4-a8e4-692e996b5515 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:48.464558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1492bd-ab3d-48f4-a8e4-692e996b5515 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:48.464881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1492bd-ab3d-48f4-a8e4-692e996b5515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:48.485291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5f1492bd-ab3d-48f4-a8e4-692e996b5515 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:48.486574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5f1492bd-ab3d-48f4-a8e4-692e996b5515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:48.495995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 961/1925] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:48.496901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1492bd-ab3d-48f4-a8e4-692e996b5515 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:48.498355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5f1492bd-ab3d-48f4-a8e4-692e996b5515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:48.547362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72848b1b00974677ad32505a5f2087bf', 'request_id': 'req-468d1a0f-711a-425a-8123-60646c701ac0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:48.554701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72848b1b00974677ad32505a5f2087bf', 'request_id': 'req-468d1a0f-711a-425a-8123-60646c701ac0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:48.572138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72848b1b00974677ad32505a5f2087bf', 'request_id': 'req-468d1a0f-711a-425a-8123-60646c701ac0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:48.630612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1492bd-ab3d-48f4-a8e4-692e996b5515 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:48.696622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] The token cache is being invalidate because project 72848b1b00974677ad32505a5f2087bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:48.697045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:48.699159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-468d1a0f-711a-425a-8123-60646c701ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:48.709638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 965/1926] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:48 2022] DELETE /identity/v3/projects/72848b1b00974677ad32505a5f2087bf => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:48.716017 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0224b0-7919-4ebe-99af-3154eb7228a4 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:48.717447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0224b0-7919-4ebe-99af-3154eb7228a4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:48.717815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0224b0-7919-4ebe-99af-3154eb7228a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:48.725366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b07f0-21e2-446c-b239-2ffc4acaba8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:48.726229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b07f0-21e2-446c-b239-2ffc4acaba8f service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:48.726716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b07f0-21e2-446c-b239-2ffc4acaba8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:48.727266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b07f0-21e2-446c-b239-2ffc4acaba8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:48.729262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-396b07f0-21e2-446c-b239-2ffc4acaba8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:48.740901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-bc0224b0-7919-4ebe-99af-3154eb7228a4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:48.742290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-bc0224b0-7919-4ebe-99af-3154eb7228a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:48.831579 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b07f0-21e2-446c-b239-2ffc4acaba8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:48.831579 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b07f0-21e2-446c-b239-2ffc4acaba8f service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:48.842706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 962/1927] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:48 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:48.844278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f871a-9d0c-41a0-8d16-47026b9c408b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:48.844765 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f871a-9d0c-41a0-8d16-47026b9c408b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:48.845688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f871a-9d0c-41a0-8d16-47026b9c408b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:48.872711 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-260f871a-9d0c-41a0-8d16-47026b9c408b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:48.874431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-260f871a-9d0c-41a0-8d16-47026b9c408b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:48.922226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 966/1928] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:48.923436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0224b0-7919-4ebe-99af-3154eb7228a4 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:48.934475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0224b0-7919-4ebe-99af-3154eb7228a4 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:48.943178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dda40-8944-4e60-bfd6-35073f3a0cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:48.943939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dda40-8944-4e60-bfd6-35073f3a0cca admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:48.944528 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dda40-8944-4e60-bfd6-35073f3a0cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:48.944897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dda40-8944-4e60-bfd6-35073f3a0cca admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:48.946289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dda40-8944-4e60-bfd6-35073f3a0cca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:48.946677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dda40-8944-4e60-bfd6-35073f3a0cca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:48.951560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dda40-8944-4e60-bfd6-35073f3a0cca admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:48.972587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 967/1929] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:48.980638 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 963/1930] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:49.000303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452f4ff-c740-4238-94ca-74efdfddaba3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.001863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452f4ff-c740-4238-94ca-74efdfddaba3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.002169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452f4ff-c740-4238-94ca-74efdfddaba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.005392 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dda40-8944-4e60-bfd6-35073f3a0cca admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.032592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dda40-8944-4e60-bfd6-35073f3a0cca admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.048088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13910b5-1def-4ec9-8032-9023f84e03c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.048088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13910b5-1def-4ec9-8032-9023f84e03c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.048088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13910b5-1def-4ec9-8032-9023f84e03c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.048088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13910b5-1def-4ec9-8032-9023f84e03c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.048413 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13910b5-1def-4ec9-8032-9023f84e03c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.048487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2452f4ff-c740-4238-94ca-74efdfddaba3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:49.048579 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2452f4ff-c740-4238-94ca-74efdfddaba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:49.051265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13910b5-1def-4ec9-8032-9023f84e03c5 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.082214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 964/1931] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:49.083073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452f4ff-c740-4238-94ca-74efdfddaba3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.086618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2452f4ff-c740-4238-94ca-74efdfddaba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:49.104045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452f4ff-c740-4238-94ca-74efdfddaba3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.108841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167629e-33be-4cfc-a328-8b1829e35575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.109727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167629e-33be-4cfc-a328-8b1829e35575 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.110420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167629e-33be-4cfc-a328-8b1829e35575 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.110869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167629e-33be-4cfc-a328-8b1829e35575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.113776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c167629e-33be-4cfc-a328-8b1829e35575 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:49.193005 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 968/1932] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:48 2022] POST /identity/v3/projects => generated 397 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:49.193005 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13910b5-1def-4ec9-8032-9023f84e03c5 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.232184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167629e-33be-4cfc-a328-8b1829e35575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.234293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167629e-33be-4cfc-a328-8b1829e35575 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.257249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 965/1933] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:49.258187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167629e-33be-4cfc-a328-8b1829e35575 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.287849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167629e-33be-4cfc-a328-8b1829e35575 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.356670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13910b5-1def-4ec9-8032-9023f84e03c5 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.366610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d02f1b-b4f4-4148-a931-10ee514912fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.367669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d02f1b-b4f4-4148-a931-10ee514912fe admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.368106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d02f1b-b4f4-4148-a931-10ee514912fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.368578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d02f1b-b4f4-4148-a931-10ee514912fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.369642 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d02f1b-b4f4-4148-a931-10ee514912fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:49.370106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d02f1b-b4f4-4148-a931-10ee514912fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.372689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d02f1b-b4f4-4148-a931-10ee514912fe admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.387443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 969/1934] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:49.394360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d02f1b-b4f4-4148-a931-10ee514912fe admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.407147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d02f1b-b4f4-4148-a931-10ee514912fe admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.413735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccdf12-0d14-489d-8d81-a55184fe2ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.414419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccdf12-0d14-489d-8d81-a55184fe2ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.414714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccdf12-0d14-489d-8d81-a55184fe2ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.414985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccdf12-0d14-489d-8d81-a55184fe2ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.415695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baccdf12-0d14-489d-8d81-a55184fe2ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.418439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baccdf12-0d14-489d-8d81-a55184fe2ff7 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.456248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbf5b5-7fe6-4268-b535-4eca08e68958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.460160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbf5b5-7fe6-4268-b535-4eca08e68958 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.460910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbf5b5-7fe6-4268-b535-4eca08e68958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.466074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbf5b5-7fe6-4268-b535-4eca08e68958 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.468405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbf5b5-7fe6-4268-b535-4eca08e68958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.471680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbf5b5-7fe6-4268-b535-4eca08e68958 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.549110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 970/1935] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:49 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:44:49.550641 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccdf12-0d14-489d-8d81-a55184fe2ff7 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.583144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 966/1936] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:49 2022] POST /identity/v3/users => generated 299 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:49.584159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbf5b5-7fe6-4268-b535-4eca08e68958 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.599703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbf5b5-7fe6-4268-b535-4eca08e68958 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.653902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccdf12-0d14-489d-8d81-a55184fe2ff7 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.660939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35b408-b301-4c45-992a-ab0e721ccd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.661942 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35b408-b301-4c45-992a-ab0e721ccd3c admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.662372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35b408-b301-4c45-992a-ab0e721ccd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.662791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35b408-b301-4c45-992a-ab0e721ccd3c admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.663643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35b408-b301-4c45-992a-ab0e721ccd3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.666344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35b408-b301-4c45-992a-ab0e721ccd3c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.685378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:49.700819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0908faf-6f78-4684-b298-f02600e97c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.701694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0908faf-6f78-4684-b298-f02600e97c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.702051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0908faf-6f78-4684-b298-f02600e97c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.702400 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0908faf-6f78-4684-b298-f02600e97c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.703409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0908faf-6f78-4684-b298-f02600e97c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.705419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0908faf-6f78-4684-b298-f02600e97c3d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.717782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 967/1937] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:49.734834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0908faf-6f78-4684-b298-f02600e97c3d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.759174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0908faf-6f78-4684-b298-f02600e97c3d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.765613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ae4d6-a5cf-4ff4-b5da-a08f147e250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.766408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ae4d6-a5cf-4ff4-b5da-a08f147e250a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.766796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ae4d6-a5cf-4ff4-b5da-a08f147e250a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.767174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ae4d6-a5cf-4ff4-b5da-a08f147e250a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.768077 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ae4d6-a5cf-4ff4-b5da-a08f147e250a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.774346 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ae4d6-a5cf-4ff4-b5da-a08f147e250a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.784647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 968/1938] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:49.789810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ae4d6-a5cf-4ff4-b5da-a08f147e250a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.812250 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ae4d6-a5cf-4ff4-b5da-a08f147e250a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.819099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bac30-c228-440d-b7c5-ef914bc2daa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.820358 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bac30-c228-440d-b7c5-ef914bc2daa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.820768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bac30-c228-440d-b7c5-ef914bc2daa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.821145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bac30-c228-440d-b7c5-ef914bc2daa5 admin admin] PATH_INFO: `/v3/projects/a34880a4c7ea44bba3f7e3b0f47cc299/users/30a65eff8f604a32a8b207b1a85f3991/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.832716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 971/1939] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:49 2022] POST /identity/v3/users => generated 288 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:49.837453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35b408-b301-4c45-992a-ab0e721ccd3c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.841273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bac30-c228-440d-b7c5-ef914bc2daa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34880a4c7ea44bba3f7e3b0f47cc299, user_id=30a65eff8f604a32a8b207b1a85f3991, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.844157 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bac30-c228-440d-b7c5-ef914bc2daa5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.848091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35b408-b301-4c45-992a-ab0e721ccd3c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.862041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc6259-eae6-46a4-bbdb-eda1dfe72115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.863064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc6259-eae6-46a4-bbdb-eda1dfe72115 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.863798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc6259-eae6-46a4-bbdb-eda1dfe72115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.865819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc6259-eae6-46a4-bbdb-eda1dfe72115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.867432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc6259-eae6-46a4-bbdb-eda1dfe72115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.873559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc6259-eae6-46a4-bbdb-eda1dfe72115 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.888404 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 972/1940] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:44:49.892720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc6259-eae6-46a4-bbdb-eda1dfe72115 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.905741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc6259-eae6-46a4-bbdb-eda1dfe72115 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:49.917453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ed647-fb44-4d56-ac65-ab1bf9f2648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:49.917453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ed647-fb44-4d56-ac65-ab1bf9f2648a admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.917453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ed647-fb44-4d56-ac65-ab1bf9f2648a admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.917453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ed647-fb44-4d56-ac65-ab1bf9f2648a admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.917809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ed647-fb44-4d56-ac65-ab1bf9f2648a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:49.917809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ed647-fb44-4d56-ac65-ab1bf9f2648a admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:49.932639 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 973/1941] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:49.936427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ed647-fb44-4d56-ac65-ab1bf9f2648a admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:49.938227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 969/1942] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:49 2022] PUT /identity/v3/projects/a34880a4c7ea44bba3f7e3b0f47cc299/users/30a65eff8f604a32a8b207b1a85f3991/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:49.952322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad9d7a-ce4c-43f7-bc3e-daf3662668b2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:49.952322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad9d7a-ce4c-43f7-bc3e-daf3662668b2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:49.952322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad9d7a-ce4c-43f7-bc3e-daf3662668b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:49.974562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ed647-fb44-4d56-ac65-ab1bf9f2648a admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:50.001503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-95ad9d7a-ce4c-43f7-bc3e-daf3662668b2 None None] MFA Rules not processed for user `30a65eff8f604a32a8b207b1a85f3991`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:50.004993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-95ad9d7a-ce4c-43f7-bc3e-daf3662668b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:50.106679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109e942-a9ec-4ef2-9f8f-a4e48818b786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:50.108516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109e942-a9ec-4ef2-9f8f-a4e48818b786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:50.109552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109e942-a9ec-4ef2-9f8f-a4e48818b786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:50.109913 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109e942-a9ec-4ef2-9f8f-a4e48818b786 admin admin] PATH_INFO: `/v3/projects/e85df18c80d54d0faf9c6cfd49586117/users/6a522270d2d94441814353fc5eaa8b61/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:50.130796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109e942-a9ec-4ef2-9f8f-a4e48818b786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85df18c80d54d0faf9c6cfd49586117, user_id=6a522270d2d94441814353fc5eaa8b61, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:50.135172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109e942-a9ec-4ef2-9f8f-a4e48818b786 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:50.175783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 970/1943] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:50.210472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 974/1944] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:49 2022] PUT /identity/v3/projects/e85df18c80d54d0faf9c6cfd49586117/users/6a522270d2d94441814353fc5eaa8b61/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:50.222351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0834649-6f31-41f0-b848-b1f8ed151f23 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:50.222642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0834649-6f31-41f0-b848-b1f8ed151f23 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:50.222925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0834649-6f31-41f0-b848-b1f8ed151f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:50.257867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f0834649-6f31-41f0-b848-b1f8ed151f23 None None] MFA Rules not processed for user `6a522270d2d94441814353fc5eaa8b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:50.259194 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f0834649-6f31-41f0-b848-b1f8ed151f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:50.366270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109e942-a9ec-4ef2-9f8f-a4e48818b786 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:50.368313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 971/1945] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:50.372921 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0834649-6f31-41f0-b848-b1f8ed151f23 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:50.457074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0834649-6f31-41f0-b848-b1f8ed151f23 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:50.462331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109e942-a9ec-4ef2-9f8f-a4e48818b786 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:50.465218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eeb9e-c92a-4412-9fbf-b4208813464b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:50.467460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eeb9e-c92a-4412-9fbf-b4208813464b admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:50.467460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eeb9e-c92a-4412-9fbf-b4208813464b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:50.467460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eeb9e-c92a-4412-9fbf-b4208813464b admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:50.467460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597eeb9e-c92a-4412-9fbf-b4208813464b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:50.471005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597eeb9e-c92a-4412-9fbf-b4208813464b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:50.472280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fa52f-23a4-4d86-b0bd-652818e4e3a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:50.472964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fa52f-23a4-4d86-b0bd-652818e4e3a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:50.473279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fa52f-23a4-4d86-b0bd-652818e4e3a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:50.473560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fa52f-23a4-4d86-b0bd-652818e4e3a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:50.522549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:50.613029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fa52f-23a4-4d86-b0bd-652818e4e3a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:50.615826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fa52f-23a4-4d86-b0bd-652818e4e3a6 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:50.640237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 972/1946] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:50 2022] POST /identity/v3/projects => generated 375 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:50.644030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eeb9e-c92a-4412-9fbf-b4208813464b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:50.648506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 975/1947] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:44:50.742804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eeb9e-c92a-4412-9fbf-b4208813464b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:50.749030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b654cb4-0f2f-40cc-8d88-62dac4f0da07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:50.749770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b654cb4-0f2f-40cc-8d88-62dac4f0da07 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:50.750127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b654cb4-0f2f-40cc-8d88-62dac4f0da07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:50.750478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b654cb4-0f2f-40cc-8d88-62dac4f0da07 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:50.751442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b654cb4-0f2f-40cc-8d88-62dac4f0da07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:50.753663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b654cb4-0f2f-40cc-8d88-62dac4f0da07 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:50.894052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 973/1948] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:50 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:50.897625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fa52f-23a4-4d86-b0bd-652818e4e3a6 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:50.915752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fa52f-23a4-4d86-b0bd-652818e4e3a6 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:50.924128 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0c988-1203-4f8e-87df-934f0cbcc7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:50.924815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0c988-1203-4f8e-87df-934f0cbcc7be admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:50.925098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0c988-1203-4f8e-87df-934f0cbcc7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:50.931290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0c988-1203-4f8e-87df-934f0cbcc7be admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:50.931290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0c988-1203-4f8e-87df-934f0cbcc7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:50.931290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0c988-1203-4f8e-87df-934f0cbcc7be admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:50.942425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 976/1949] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:44:50.945838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b654cb4-0f2f-40cc-8d88-62dac4f0da07 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:50.957715 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b654cb4-0f2f-40cc-8d88-62dac4f0da07 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:50.964993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32884ac-6d6c-4719-8102-b3fdb14ae2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:50.965826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32884ac-6d6c-4719-8102-b3fdb14ae2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:50.966189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32884ac-6d6c-4719-8102-b3fdb14ae2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:50.966595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32884ac-6d6c-4719-8102-b3fdb14ae2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:50.967424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32884ac-6d6c-4719-8102-b3fdb14ae2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:50.969294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32884ac-6d6c-4719-8102-b3fdb14ae2a0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:50.989715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0c988-1203-4f8e-87df-934f0cbcc7be admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:51.003245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 974/1950] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:51.011325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0c988-1203-4f8e-87df-934f0cbcc7be admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:51.016841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969017fc-6683-44dc-84be-40ee1057a14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:51.018113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969017fc-6683-44dc-84be-40ee1057a14e admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:51.018544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969017fc-6683-44dc-84be-40ee1057a14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:51.018993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969017fc-6683-44dc-84be-40ee1057a14e admin admin] PATH_INFO: `/v3/projects/6f8dd37346fe459fb9f643505d53f746/users/113b470153c944a282dd2104fc7c30a8/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:51.037929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969017fc-6683-44dc-84be-40ee1057a14e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f8dd37346fe459fb9f643505d53f746, user_id=113b470153c944a282dd2104fc7c30a8, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:51.041821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969017fc-6683-44dc-84be-40ee1057a14e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:51.113213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 977/1951] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:50 2022] PUT /identity/v3/projects/6f8dd37346fe459fb9f643505d53f746/users/113b470153c944a282dd2104fc7c30a8/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:51.121324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5cca-b8cc-4b7f-b1af-be7ca401de9d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:51.121616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5cca-b8cc-4b7f-b1af-be7ca401de9d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:51.122024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5cca-b8cc-4b7f-b1af-be7ca401de9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:51.152141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-278b5cca-b8cc-4b7f-b1af-be7ca401de9d None None] MFA Rules not processed for user `113b470153c944a282dd2104fc7c30a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:51.153344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-278b5cca-b8cc-4b7f-b1af-be7ca401de9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:51.229126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 975/1952] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:51.238616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969017fc-6683-44dc-84be-40ee1057a14e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:51.348292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969017fc-6683-44dc-84be-40ee1057a14e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:51.353481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c960b-ce7f-4231-a6ce-185aba4264e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:51.354326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c960b-ce7f-4231-a6ce-185aba4264e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:51.354622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c960b-ce7f-4231-a6ce-185aba4264e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:51.354973 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c960b-ce7f-4231-a6ce-185aba4264e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:51.355748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c960b-ce7f-4231-a6ce-185aba4264e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:51.364497 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c960b-ce7f-4231-a6ce-185aba4264e7 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:51.585624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b5cca-b8cc-4b7f-b1af-be7ca401de9d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:51.586335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 978/1953] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:51 2022] POST /identity/v3/projects => generated 375 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:51.613283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b5cca-b8cc-4b7f-b1af-be7ca401de9d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:51.678361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681daf58-e33b-4d5e-ac8a-3438539c0cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:51.679598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681daf58-e33b-4d5e-ac8a-3438539c0cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:51.679598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681daf58-e33b-4d5e-ac8a-3438539c0cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:51.679598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681daf58-e33b-4d5e-ac8a-3438539c0cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:51.680333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681daf58-e33b-4d5e-ac8a-3438539c0cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:51.682998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681daf58-e33b-4d5e-ac8a-3438539c0cf7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:51.805291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 976/1954] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:51 2022] POST /identity/v3/users => generated 294 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:44:51.810568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:51.814611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c960b-ce7f-4231-a6ce-185aba4264e7 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:51.828357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c960b-ce7f-4231-a6ce-185aba4264e7 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:51.843500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fd251-42e5-48d5-910f-02f849ebf7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:51.844198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fd251-42e5-48d5-910f-02f849ebf7bc admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:51.844485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fd251-42e5-48d5-910f-02f849ebf7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:51.844761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fd251-42e5-48d5-910f-02f849ebf7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:51.845743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fd251-42e5-48d5-910f-02f849ebf7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:51.847580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fd251-42e5-48d5-910f-02f849ebf7bc admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:51.874537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 979/1955] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:51.876837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681daf58-e33b-4d5e-ac8a-3438539c0cf7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:51.889634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681daf58-e33b-4d5e-ac8a-3438539c0cf7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:51.903186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa9063-3513-470b-9d35-9f0ec63e69d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:51.904116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa9063-3513-470b-9d35-9f0ec63e69d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:51.904403 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa9063-3513-470b-9d35-9f0ec63e69d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:51.904684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa9063-3513-470b-9d35-9f0ec63e69d8 admin admin] PATH_INFO: `/v3/projects/6e078a3c3baa4c539ba3bbb2ee21de33/users/0c69bda44d7d4842a4a3c4ddd7918249/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:51.937616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa9063-3513-470b-9d35-9f0ec63e69d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e078a3c3baa4c539ba3bbb2ee21de33, user_id=0c69bda44d7d4842a4a3c4ddd7918249, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:51.940287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa9063-3513-470b-9d35-9f0ec63e69d8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:52.035195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 977/1956] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:51 2022] PUT /identity/v3/projects/6e078a3c3baa4c539ba3bbb2ee21de33/users/0c69bda44d7d4842a4a3c4ddd7918249/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:44:52.036354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fd251-42e5-48d5-910f-02f849ebf7bc admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:52.048135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fd251-42e5-48d5-910f-02f849ebf7bc admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:52.127772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217e9bb-a675-49a4-b988-f3ddb94f5bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:52.128570 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217e9bb-a675-49a4-b988-f3ddb94f5bef admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:52.128862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217e9bb-a675-49a4-b988-f3ddb94f5bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:52.129142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217e9bb-a675-49a4-b988-f3ddb94f5bef admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:52.130002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217e9bb-a675-49a4-b988-f3ddb94f5bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:52.131992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217e9bb-a675-49a4-b988-f3ddb94f5bef admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:52.148176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 980/1957] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:52.149743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa9063-3513-470b-9d35-9f0ec63e69d8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:52.166782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa9063-3513-470b-9d35-9f0ec63e69d8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:52.172358 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebf1c7-896d-4898-8886-f7c541a7dfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:52.173280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebf1c7-896d-4898-8886-f7c541a7dfb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:52.173672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebf1c7-896d-4898-8886-f7c541a7dfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:52.173957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebf1c7-896d-4898-8886-f7c541a7dfb1 admin admin] PATH_INFO: `/v3/projects/6e078a3c3baa4c539ba3bbb2ee21de33/users/0c69bda44d7d4842a4a3c4ddd7918249/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:52.175971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebf1c7-896d-4898-8886-f7c541a7dfb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e078a3c3baa4c539ba3bbb2ee21de33, user_id=0c69bda44d7d4842a4a3c4ddd7918249, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:52.178745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebf1c7-896d-4898-8886-f7c541a7dfb1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:52.203880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 978/1958] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:52 2022] PUT /identity/v3/projects/6e078a3c3baa4c539ba3bbb2ee21de33/users/0c69bda44d7d4842a4a3c4ddd7918249/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:52.216516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315605f4-442b-4357-a9f9-c83c9a35f37d None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:52.216516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315605f4-442b-4357-a9f9-c83c9a35f37d None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:52.216762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315605f4-442b-4357-a9f9-c83c9a35f37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:52.246488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-315605f4-442b-4357-a9f9-c83c9a35f37d None None] MFA Rules not processed for user `0c69bda44d7d4842a4a3c4ddd7918249`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:52.247735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-315605f4-442b-4357-a9f9-c83c9a35f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:52.359978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 981/1959] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:52.403137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebf1c7-896d-4898-8886-f7c541a7dfb1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:52.403137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-62ebf1c7-896d-4898-8886-f7c541a7dfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:52.470585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebf1c7-896d-4898-8886-f7c541a7dfb1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:52.476495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec91c6-75bd-48e4-89af-bdbd5fdba2fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:52.477877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec91c6-75bd-48e4-89af-bdbd5fdba2fb service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:52.478221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec91c6-75bd-48e4-89af-bdbd5fdba2fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:52.478509 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec91c6-75bd-48e4-89af-bdbd5fdba2fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:52.498623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:52.599520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec91c6-75bd-48e4-89af-bdbd5fdba2fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:52.600986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec91c6-75bd-48e4-89af-bdbd5fdba2fb service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:52.623637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 979/1960] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:53.019642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:54.771848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:56.140861 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315605f4-442b-4357-a9f9-c83c9a35f37d None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:56.142328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-315605f4-442b-4357-a9f9-c83c9a35f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:56.200079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:56.214360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315605f4-442b-4357-a9f9-c83c9a35f37d None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:56.219065 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:56.219929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:56.220210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:56.220482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] PATH_INFO: `/v3/users/4c371f62facd40bc96e98a3e0e61e2f6` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:56.221689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c371f62facd40bc96e98a3e0e61e2f6)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:56.223598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:56.463741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c371f62facd40bc96e98a3e0e61e2f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:56.464498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c371f62facd40bc96e98a3e0e61e2f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:56.489333 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c371f62facd40bc96e98a3e0e61e2f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:56.499493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c371f62facd40bc96e98a3e0e61e2f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:56.579341 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c371f62facd40bc96e98a3e0e61e2f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:56.586130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 982/1961] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:56 2022] DELETE /identity/v3/users/4c371f62facd40bc96e98a3e0e61e2f6 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:56.590763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62449ea8-1834-4700-aef1-22b493627b94 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:56.591129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62449ea8-1834-4700-aef1-22b493627b94 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:56.591830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62449ea8-1834-4700-aef1-22b493627b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:56.612149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-62449ea8-1834-4700-aef1-22b493627b94 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:56.615011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-62449ea8-1834-4700-aef1-22b493627b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:56.692617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 980/1962] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:56.702651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:56.704120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:56.769575 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc79db-b100-4df4-9822-ba00d980141f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:56.773281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be08a10-d3bf-4271-a37f-d40e3a133844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:56.774212 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be08a10-d3bf-4271-a37f-d40e3a133844 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:56.774524 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be08a10-d3bf-4271-a37f-d40e3a133844 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:56.774802 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be08a10-d3bf-4271-a37f-d40e3a133844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:56.785887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be08a10-d3bf-4271-a37f-d40e3a133844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:56.787245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be08a10-d3bf-4271-a37f-d40e3a133844 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:56.799798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 983/1963] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:57.173968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:57.286895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62449ea8-1834-4700-aef1-22b493627b94 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:57.292163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-62449ea8-1834-4700-aef1-22b493627b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:57.319428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62449ea8-1834-4700-aef1-22b493627b94 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:57.328763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:57.330543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:57.330543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:57.330543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] PATH_INFO: `/v3/projects/5995536198644c6099cb9a5ebf8f7b27` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:57.331668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5995536198644c6099cb9a5ebf8f7b27)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:57.333614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:57.474751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5995536198644c6099cb9a5ebf8f7b27', 'request_id': 'req-9c41a329-c468-4092-ab5a-8852444bc75b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:57.555557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5995536198644c6099cb9a5ebf8f7b27', 'request_id': 'req-9c41a329-c468-4092-ab5a-8852444bc75b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:57.560101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5995536198644c6099cb9a5ebf8f7b27', 'request_id': 'req-9c41a329-c468-4092-ab5a-8852444bc75b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:57.572321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] The token cache is being invalidate because project 5995536198644c6099cb9a5ebf8f7b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:57.572610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:57.573880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:57.578609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 981/1964] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:57 2022] DELETE /identity/v3/projects/5995536198644c6099cb9a5ebf8f7b27 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:57.580062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be08a10-d3bf-4271-a37f-d40e3a133844 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:57.591523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be08a10-d3bf-4271-a37f-d40e3a133844 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:57.652468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:57.671912 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:57.674224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:57.674224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:57.674532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] PATH_INFO: `/v3/users/4dc2b3e9b84440f594a462955b7aed40` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:57.676190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc2b3e9b84440f594a462955b7aed40)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:57.678387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:57.955964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc2b3e9b84440f594a462955b7aed40'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:57.956628 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc2b3e9b84440f594a462955b7aed40'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:57.968909 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc2b3e9b84440f594a462955b7aed40'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:57.976188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc2b3e9b84440f594a462955b7aed40'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:58.136256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc2b3e9b84440f594a462955b7aed40'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:58.142928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 984/1965] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:44:57 2022] DELETE /identity/v3/users/4dc2b3e9b84440f594a462955b7aed40 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:58.288117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:58.619159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:58.621881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:58.647402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41a329-c468-4092-ab5a-8852444bc75b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:58.671340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:58.722994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:58.723857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:58.724137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:58.724410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] PATH_INFO: `/v3/projects/1506bdca26c14b14953c304c005bace5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:58.725616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1506bdca26c14b14953c304c005bace5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:58.727498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:58.859489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1506bdca26c14b14953c304c005bace5', 'request_id': 'req-55c8c204-111d-406a-a6ee-5b55199d5de6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:58.910617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:58.912071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:58.950452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1506bdca26c14b14953c304c005bace5', 'request_id': 'req-55c8c204-111d-406a-a6ee-5b55199d5de6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:58.956910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1506bdca26c14b14953c304c005bace5', 'request_id': 'req-55c8c204-111d-406a-a6ee-5b55199d5de6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:58.968763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] The token cache is being invalidate because project 1506bdca26c14b14953c304c005bace5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:44:58.972920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:58.972920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-55c8c204-111d-406a-a6ee-5b55199d5de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:44:58.972920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 982/1966] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:44:58 2022] DELETE /identity/v3/projects/1506bdca26c14b14953c304c005bace5 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:58.973224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e65817-7265-4f87-905a-b7ecb61775ca admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:58.976547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f1a7f-989a-4f58-ad69-7af36b495046 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:58.976901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f1a7f-989a-4f58-ad69-7af36b495046 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:58.977283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f1a7f-989a-4f58-ad69-7af36b495046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:58.996716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-051f1a7f-989a-4f58-ad69-7af36b495046 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:58.998115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-051f1a7f-989a-4f58-ad69-7af36b495046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:59.000501 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:59.026683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565dd5f-fd8e-4a51-b6e4-97e1809385aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:59.027445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565dd5f-fd8e-4a51-b6e4-97e1809385aa service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:59.027737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565dd5f-fd8e-4a51-b6e4-97e1809385aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:59.028012 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565dd5f-fd8e-4a51-b6e4-97e1809385aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:59.029838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6565dd5f-fd8e-4a51-b6e4-97e1809385aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:59.059537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 983/1967] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:59.067740 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ec183-0d62-4f90-9672-a7cfb11f38e5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:59.068032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ec183-0d62-4f90-9672-a7cfb11f38e5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:59.068311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ec183-0d62-4f90-9672-a7cfb11f38e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:59.086511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-036ec183-0d62-4f90-9672-a7cfb11f38e5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:59.090985 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-036ec183-0d62-4f90-9672-a7cfb11f38e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:59.096011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565dd5f-fd8e-4a51-b6e4-97e1809385aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:59.096011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565dd5f-fd8e-4a51-b6e4-97e1809385aa service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:59.102709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 985/1968] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:44:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:44:59.105958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 984/1969] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:44:59.115936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565dd5f-fd8e-4a51-b6e4-97e1809385aa service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:59.133347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565dd5f-fd8e-4a51-b6e4-97e1809385aa service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:59.137017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:59.140368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8abb7-161c-4082-b085-94c61bfe6ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:59.141261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8abb7-161c-4082-b085-94c61bfe6ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:59.141811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8abb7-161c-4082-b085-94c61bfe6ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:59.145775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8abb7-161c-4082-b085-94c61bfe6ac7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:59.145775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8abb7-161c-4082-b085-94c61bfe6ac7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:44:59.145775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8abb7-161c-4082-b085-94c61bfe6ac7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:59.147810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8abb7-161c-4082-b085-94c61bfe6ac7 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:59.177927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 986/1970] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:44:59.180683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ec183-0d62-4f90-9672-a7cfb11f38e5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:59.190944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ec183-0d62-4f90-9672-a7cfb11f38e5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:59.196117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe792e-413b-4d40-a20b-bbd569d51df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:59.196828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe792e-413b-4d40-a20b-bbd569d51df6 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:59.197119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe792e-413b-4d40-a20b-bbd569d51df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:59.197794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe792e-413b-4d40-a20b-bbd569d51df6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:59.198637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe792e-413b-4d40-a20b-bbd569d51df6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:59.200556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe792e-413b-4d40-a20b-bbd569d51df6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:59.332845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 985/1971] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:59 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:59.336968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8abb7-161c-4082-b085-94c61bfe6ac7 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:59.348177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8abb7-161c-4082-b085-94c61bfe6ac7 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:59.409587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072aef2c-ad7c-4f19-88c4-f0c0a7c614c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:59.410257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072aef2c-ad7c-4f19-88c4-f0c0a7c614c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:59.411086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072aef2c-ad7c-4f19-88c4-f0c0a7c614c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:59.411086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072aef2c-ad7c-4f19-88c4-f0c0a7c614c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:59.411502 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072aef2c-ad7c-4f19-88c4-f0c0a7c614c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:59.413511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072aef2c-ad7c-4f19-88c4-f0c0a7c614c0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:59.436627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:59.552360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 987/1972] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:44:59 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:44:59.556841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe792e-413b-4d40-a20b-bbd569d51df6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:59.567490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe792e-413b-4d40-a20b-bbd569d51df6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:59.572185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a1f3-2f79-4d70-8fda-966c8cc3f4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:59.573734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a1f3-2f79-4d70-8fda-966c8cc3f4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:59.573734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a1f3-2f79-4d70-8fda-966c8cc3f4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:59.573943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a1f3-2f79-4d70-8fda-966c8cc3f4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:59.575012 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039a1f3-2f79-4d70-8fda-966c8cc3f4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:59.576607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039a1f3-2f79-4d70-8fda-966c8cc3f4e8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:59.587015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 986/1973] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:59.591862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072aef2c-ad7c-4f19-88c4-f0c0a7c614c0 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:59.602128 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072aef2c-ad7c-4f19-88c4-f0c0a7c614c0 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:59.606892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffb3c1-803a-46a3-80a7-c2d90810ac93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:59.607566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffb3c1-803a-46a3-80a7-c2d90810ac93 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:59.607851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffb3c1-803a-46a3-80a7-c2d90810ac93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:59.608127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffb3c1-803a-46a3-80a7-c2d90810ac93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:59.608869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffb3c1-803a-46a3-80a7-c2d90810ac93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:59.610616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffb3c1-803a-46a3-80a7-c2d90810ac93 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:59.622751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 988/1974] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:44:59.642741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a1f3-2f79-4d70-8fda-966c8cc3f4e8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:44:59.653884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a1f3-2f79-4d70-8fda-966c8cc3f4e8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:44:59.658769 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a7541-a1be-4ca8-9ceb-2618428f5ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:44:59.659751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a7541-a1be-4ca8-9ceb-2618428f5ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:59.660106 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a7541-a1be-4ca8-9ceb-2618428f5ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:59.660453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a7541-a1be-4ca8-9ceb-2618428f5ebe admin admin] PATH_INFO: `/v3/projects/ece2edfd519a4fe78a221681f87c2a62/users/6b86eed4b4394339bc11072444152c8b/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:59.676162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a7541-a1be-4ca8-9ceb-2618428f5ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece2edfd519a4fe78a221681f87c2a62, user_id=6b86eed4b4394339bc11072444152c8b, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:44:59.696111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a7541-a1be-4ca8-9ceb-2618428f5ebe admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:44:59.801303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:44:59.823922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 987/1975] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:44:59 2022] PUT /identity/v3/projects/ece2edfd519a4fe78a221681f87c2a62/users/6b86eed4b4394339bc11072444152c8b/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:44:59.834457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a493b-44d2-4a66-b593-27b1b95bde41 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:44:59.834884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a493b-44d2-4a66-b593-27b1b95bde41 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:44:59.835257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a493b-44d2-4a66-b593-27b1b95bde41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:44:59.865035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-070a493b-44d2-4a66-b593-27b1b95bde41 None None] MFA Rules not processed for user `6b86eed4b4394339bc11072444152c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:44:59.866328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-070a493b-44d2-4a66-b593-27b1b95bde41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:44:59.956687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 989/1976] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:44:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:44:59.961522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a7541-a1be-4ca8-9ceb-2618428f5ebe admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:00.034192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a7541-a1be-4ca8-9ceb-2618428f5ebe admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:00.038961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959eefc-6667-4c2d-bfba-177c213600de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:00.039620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959eefc-6667-4c2d-bfba-177c213600de admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:00.039915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959eefc-6667-4c2d-bfba-177c213600de admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:00.040183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959eefc-6667-4c2d-bfba-177c213600de admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:00.040876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959eefc-6667-4c2d-bfba-177c213600de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:00.043554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959eefc-6667-4c2d-bfba-177c213600de admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:00.054939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:00.239226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 988/1977] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:44:59 2022] POST /identity/v3/projects => generated 387 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:00.245562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a493b-44d2-4a66-b593-27b1b95bde41 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:00.270156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959eefc-6667-4c2d-bfba-177c213600de admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:00.272000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f959eefc-6667-4c2d-bfba-177c213600de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:00.273603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a493b-44d2-4a66-b593-27b1b95bde41 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:00.354555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959eefc-6667-4c2d-bfba-177c213600de admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:00.359483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:00.359693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96333c77-83ab-4065-b9a9-a92e0e299ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:00.360219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96333c77-83ab-4065-b9a9-a92e0e299ded admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:00.360369 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:00.360501 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96333c77-83ab-4065-b9a9-a92e0e299ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:00.360650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:00.360757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96333c77-83ab-4065-b9a9-a92e0e299ded admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:00.360930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] PATH_INFO: `/v3/users/30a65eff8f604a32a8b207b1a85f3991` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:00.361553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96333c77-83ab-4065-b9a9-a92e0e299ded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:00.362211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] RBAC: Authorizing `identity:delete_user(user_id=30a65eff8f604a32a8b207b1a85f3991)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:00.363439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96333c77-83ab-4065-b9a9-a92e0e299ded admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:00.364211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:00.484579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 990/1978] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:00 2022] POST /identity/v3/users => generated 294 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:45:00.487891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96333c77-83ab-4065-b9a9-a92e0e299ded admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:00.491642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:00.500121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96333c77-83ab-4065-b9a9-a92e0e299ded admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:00.505108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cddbe-ca63-484d-9510-1f19f3f4c31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:00.505817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cddbe-ca63-484d-9510-1f19f3f4c31d admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:00.506145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cddbe-ca63-484d-9510-1f19f3f4c31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:00.506409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cddbe-ca63-484d-9510-1f19f3f4c31d admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:00.507149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5cddbe-ca63-484d-9510-1f19f3f4c31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:00.508893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5cddbe-ca63-484d-9510-1f19f3f4c31d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:00.519488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 991/1979] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:45:00.523466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cddbe-ca63-484d-9510-1f19f3f4c31d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:00.534334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cddbe-ca63-484d-9510-1f19f3f4c31d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:00.539216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d896592-49e5-40c1-a8dd-a9b557953934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:00.542259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d896592-49e5-40c1-a8dd-a9b557953934 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:00.542259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d896592-49e5-40c1-a8dd-a9b557953934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:00.542259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d896592-49e5-40c1-a8dd-a9b557953934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:00.542259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d896592-49e5-40c1-a8dd-a9b557953934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:00.543339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d896592-49e5-40c1-a8dd-a9b557953934 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:00.554479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 992/1980] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:45:00.558060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d896592-49e5-40c1-a8dd-a9b557953934 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:00.567977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d896592-49e5-40c1-a8dd-a9b557953934 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:00.572653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70743515-ba93-49f6-ae35-8dd1fc138fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:00.573577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70743515-ba93-49f6-ae35-8dd1fc138fda admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:00.573890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70743515-ba93-49f6-ae35-8dd1fc138fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:00.574128 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70743515-ba93-49f6-ae35-8dd1fc138fda admin admin] PATH_INFO: `/v3/projects/8736e59ad3aa4de5875955ae82d06917/users/3c162fae2a0d4e8c81a2808e4f811fc8/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:00.589648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70743515-ba93-49f6-ae35-8dd1fc138fda admin admin] RBAC: Authorizing `identity:create_grant(project_id=8736e59ad3aa4de5875955ae82d06917, user_id=3c162fae2a0d4e8c81a2808e4f811fc8, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:00.592325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70743515-ba93-49f6-ae35-8dd1fc138fda admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:00.617322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30a65eff8f604a32a8b207b1a85f3991'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:00.626107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30a65eff8f604a32a8b207b1a85f3991'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:00.639039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30a65eff8f604a32a8b207b1a85f3991'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:00.680061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 993/1981] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:00 2022] PUT /identity/v3/projects/8736e59ad3aa4de5875955ae82d06917/users/3c162fae2a0d4e8c81a2808e4f811fc8/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:00.692143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7ac60-d9f6-4ca6-8dfa-b8f82b9f2311 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:00.692428 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7ac60-d9f6-4ca6-8dfa-b8f82b9f2311 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:00.692871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7ac60-d9f6-4ca6-8dfa-b8f82b9f2311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:00.706965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30a65eff8f604a32a8b207b1a85f3991'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:00.707966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-87540b2b-c250-4483-9ad1-77e68a19200a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30a65eff8f604a32a8b207b1a85f3991'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:00.715560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 989/1982] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:00 2022] DELETE /identity/v3/users/30a65eff8f604a32a8b207b1a85f3991 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:00.720238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb4cce-d6c4-4472-9db5-ca4442c6fe87 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:00.720750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb4cce-d6c4-4472-9db5-ca4442c6fe87 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:00.721228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb4cce-d6c4-4472-9db5-ca4442c6fe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:00.752328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-2cd7ac60-d9f6-4ca6-8dfa-b8f82b9f2311 None None] MFA Rules not processed for user `3c162fae2a0d4e8c81a2808e4f811fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:00.754915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2cd7ac60-d9f6-4ca6-8dfa-b8f82b9f2311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:00.762961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f3cb4cce-d6c4-4472-9db5-ca4442c6fe87 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:00.764703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f3cb4cce-d6c4-4472-9db5-ca4442c6fe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:00.847646 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 990/1983] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:45:00.858817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb4cce-d6c4-4472-9db5-ca4442c6fe87 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:00.860518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f3cb4cce-d6c4-4472-9db5-ca4442c6fe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:00.899100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 994/1984] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:45:00.904298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7ac60-d9f6-4ca6-8dfa-b8f82b9f2311 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:00.905893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2cd7ac60-d9f6-4ca6-8dfa-b8f82b9f2311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:00.918330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:00.924386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7ac60-d9f6-4ca6-8dfa-b8f82b9f2311 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:00.931848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c783c51-1f59-4fee-8d2e-2021f132a95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:00.933120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c783c51-1f59-4fee-8d2e-2021f132a95c admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:00.933637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c783c51-1f59-4fee-8d2e-2021f132a95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:00.934149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c783c51-1f59-4fee-8d2e-2021f132a95c admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:00.935558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c783c51-1f59-4fee-8d2e-2021f132a95c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:00.939242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c783c51-1f59-4fee-8d2e-2021f132a95c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:00.943707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb4cce-d6c4-4472-9db5-ca4442c6fe87 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:00.948441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176a46f-d240-49e5-8070-3838b1d3f4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:00.949273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a46f-d240-49e5-8070-3838b1d3f4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:00.949594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a46f-d240-49e5-8070-3838b1d3f4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:00.954058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a46f-d240-49e5-8070-3838b1d3f4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:00.959973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176a46f-d240-49e5-8070-3838b1d3f4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:00.961619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176a46f-d240-49e5-8070-3838b1d3f4a1 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:00.972987 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 991/1985] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:45:01.062815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 995/1986] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:00 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:45:01.064163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176a46f-d240-49e5-8070-3838b1d3f4a1 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:01.076089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176a46f-d240-49e5-8070-3838b1d3f4a1 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:01.143142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fee70-b583-4c14-a3d0-9b63617fdadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:01.143839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fee70-b583-4c14-a3d0-9b63617fdadb admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:01.144120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fee70-b583-4c14-a3d0-9b63617fdadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:01.144437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fee70-b583-4c14-a3d0-9b63617fdadb admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:01.149687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fee70-b583-4c14-a3d0-9b63617fdadb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:01.159608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fee70-b583-4c14-a3d0-9b63617fdadb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:01.160942 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c783c51-1f59-4fee-8d2e-2021f132a95c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:01.163040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1c783c51-1f59-4fee-8d2e-2021f132a95c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:01.233656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:01.247868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c783c51-1f59-4fee-8d2e-2021f132a95c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:01.251864 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438dd3d-5dd8-4a86-bf0a-5ca4ed772674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:01.252536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438dd3d-5dd8-4a86-bf0a-5ca4ed772674 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:01.252822 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438dd3d-5dd8-4a86-bf0a-5ca4ed772674 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:01.253103 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438dd3d-5dd8-4a86-bf0a-5ca4ed772674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:01.254996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8438dd3d-5dd8-4a86-bf0a-5ca4ed772674 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:01.328221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438dd3d-5dd8-4a86-bf0a-5ca4ed772674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:01.330239 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438dd3d-5dd8-4a86-bf0a-5ca4ed772674 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:01.335340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 992/1987] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:01 2022] POST /identity/v3/users => generated 299 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:01.346807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 996/1988] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:45:01.347006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fee70-b583-4c14-a3d0-9b63617fdadb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:01.362407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fee70-b583-4c14-a3d0-9b63617fdadb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:01.367927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf9851-0889-4315-9c80-8eb2c02afc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:01.368774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf9851-0889-4315-9c80-8eb2c02afc49 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:01.369139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf9851-0889-4315-9c80-8eb2c02afc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:01.369781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf9851-0889-4315-9c80-8eb2c02afc49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:01.370689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf9851-0889-4315-9c80-8eb2c02afc49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:01.372423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf9851-0889-4315-9c80-8eb2c02afc49 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:01.382658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:01.387049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 993/1989] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:01.392489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438dd3d-5dd8-4a86-bf0a-5ca4ed772674 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:01.404140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438dd3d-5dd8-4a86-bf0a-5ca4ed772674 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:01.409050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee7c57-44e4-4da4-95d3-4a56b8213aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:01.409980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee7c57-44e4-4da4-95d3-4a56b8213aef admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:01.410267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee7c57-44e4-4da4-95d3-4a56b8213aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:01.410545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee7c57-44e4-4da4-95d3-4a56b8213aef admin admin] PATH_INFO: `/v3/projects/9d80da6629424d66b3cff407b10d9836/users/73dc177b67824d1d9e3095dcb2793687/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:01.426623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee7c57-44e4-4da4-95d3-4a56b8213aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d80da6629424d66b3cff407b10d9836, user_id=73dc177b67824d1d9e3095dcb2793687, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:01.429312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee7c57-44e4-4da4-95d3-4a56b8213aef admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:01.499777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf9851-0889-4315-9c80-8eb2c02afc49 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:01.502059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2ddf9851-0889-4315-9c80-8eb2c02afc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:01.524394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf9851-0889-4315-9c80-8eb2c02afc49 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:01.538848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 997/1990] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:01 2022] PUT /identity/v3/projects/9d80da6629424d66b3cff407b10d9836/users/73dc177b67824d1d9e3095dcb2793687/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:01.545481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee7c57-44e4-4da4-95d3-4a56b8213aef admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:01.612605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:01.612605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:01.612605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:01.613052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] PATH_INFO: `/v3/projects/a34880a4c7ea44bba3f7e3b0f47cc299` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:01.614590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a34880a4c7ea44bba3f7e3b0f47cc299)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:01.617632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:01.657433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee7c57-44e4-4da4-95d3-4a56b8213aef admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:01.663854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72a9a5-c515-4b18-9e5e-a1f261a6deaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:01.664640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72a9a5-c515-4b18-9e5e-a1f261a6deaa admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:01.664991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72a9a5-c515-4b18-9e5e-a1f261a6deaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:01.665396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72a9a5-c515-4b18-9e5e-a1f261a6deaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:01.666203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72a9a5-c515-4b18-9e5e-a1f261a6deaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:01.667921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72a9a5-c515-4b18-9e5e-a1f261a6deaa admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:01.678962 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 998/1991] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:45:01.683210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72a9a5-c515-4b18-9e5e-a1f261a6deaa admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:01.693289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72a9a5-c515-4b18-9e5e-a1f261a6deaa admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:01.699656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71826e1f-7add-4add-8de2-9dd2de08014c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:01.700567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71826e1f-7add-4add-8de2-9dd2de08014c admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:01.704957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71826e1f-7add-4add-8de2-9dd2de08014c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:01.704957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71826e1f-7add-4add-8de2-9dd2de08014c admin admin] PATH_INFO: `/v3/projects/9d80da6629424d66b3cff407b10d9836/users/73dc177b67824d1d9e3095dcb2793687/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:01.704957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71826e1f-7add-4add-8de2-9dd2de08014c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d80da6629424d66b3cff407b10d9836, user_id=73dc177b67824d1d9e3095dcb2793687, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:01.705999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71826e1f-7add-4add-8de2-9dd2de08014c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:01.718367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 999/1992] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:01 2022] PUT /identity/v3/projects/9d80da6629424d66b3cff407b10d9836/users/73dc177b67824d1d9e3095dcb2793687/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:01.728045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca9a2e-07ec-435f-9c42-07bc40c2e258 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:01.728336 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca9a2e-07ec-435f-9c42-07bc40c2e258 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:01.728613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca9a2e-07ec-435f-9c42-07bc40c2e258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:01.756885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a34880a4c7ea44bba3f7e3b0f47cc299', 'request_id': 'req-b0749357-24b2-410e-a013-8ebe0e180909'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:01.757536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-a4ca9a2e-07ec-435f-9c42-07bc40c2e258 None None] MFA Rules not processed for user `73dc177b67824d1d9e3095dcb2793687`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:01.758778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a4ca9a2e-07ec-435f-9c42-07bc40c2e258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:01.827717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a34880a4c7ea44bba3f7e3b0f47cc299', 'request_id': 'req-b0749357-24b2-410e-a013-8ebe0e180909'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:01.829028 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1000/1993] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:45:01.834748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a34880a4c7ea44bba3f7e3b0f47cc299', 'request_id': 'req-b0749357-24b2-410e-a013-8ebe0e180909'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:01.840165 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca9a2e-07ec-435f-9c42-07bc40c2e258 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:01.840339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:01.850120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] The token cache is being invalidate because project a34880a4c7ea44bba3f7e3b0f47cc299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:01.850327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:01.850965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:01.852535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 994/1994] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:01 2022] DELETE /identity/v3/projects/a34880a4c7ea44bba3f7e3b0f47cc299 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:01.904115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca9a2e-07ec-435f-9c42-07bc40c2e258 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:01.908507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735484d1-201d-4744-8dec-9fad7cfd1fdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:01.911125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735484d1-201d-4744-8dec-9fad7cfd1fdd service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:01.911125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735484d1-201d-4744-8dec-9fad7cfd1fdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:01.911125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735484d1-201d-4744-8dec-9fad7cfd1fdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:01.912528 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-735484d1-201d-4744-8dec-9fad7cfd1fdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:01.927276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735484d1-201d-4744-8dec-9fad7cfd1fdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:01.929406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735484d1-201d-4744-8dec-9fad7cfd1fdd service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:01.941541 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1001/1995] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:45:02.125601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:02.617072 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:02.705245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:03.067828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:03.147615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:03.149590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:03.162421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0749357-24b2-410e-a013-8ebe0e180909 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:03.166003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5e43a-7ce8-45db-8389-39c1be0cbf19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:03.166777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5e43a-7ce8-45db-8389-39c1be0cbf19 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:03.167106 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5e43a-7ce8-45db-8389-39c1be0cbf19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:03.167388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5e43a-7ce8-45db-8389-39c1be0cbf19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:03.169231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-16a5e43a-7ce8-45db-8389-39c1be0cbf19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:03.227842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5e43a-7ce8-45db-8389-39c1be0cbf19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:03.229652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5e43a-7ce8-45db-8389-39c1be0cbf19 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:03.239956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 995/1996] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:03.545366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:03.817252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:04.348293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:04.379839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:04.889324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:04.936741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:05.414566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:05.488599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:06.065196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:06.208073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:06.379867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:06.469914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735484d1-201d-4744-8dec-9fad7cfd1fdd service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:06.471362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-735484d1-201d-4744-8dec-9fad7cfd1fdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:06.557994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735484d1-201d-4744-8dec-9fad7cfd1fdd service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:06.565326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:06.566318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:06.566674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:06.567130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] PATH_INFO: `/v3/users/6a522270d2d94441814353fc5eaa8b61` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:06.568367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a522270d2d94441814353fc5eaa8b61)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:06.570573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:06.817176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a522270d2d94441814353fc5eaa8b61'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:06.817875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a522270d2d94441814353fc5eaa8b61'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:06.832289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a522270d2d94441814353fc5eaa8b61'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:06.840611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a522270d2d94441814353fc5eaa8b61'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:06.941560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a522270d2d94441814353fc5eaa8b61'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:06.948553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1002/1997] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:06 2022] DELETE /identity/v3/users/6a522270d2d94441814353fc5eaa8b61 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:06.953978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb439b11-7c34-4dea-a258-41c622e6264f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:06.954448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb439b11-7c34-4dea-a258-41c622e6264f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:06.954563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb439b11-7c34-4dea-a258-41c622e6264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:06.973536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-eb439b11-7c34-4dea-a258-41c622e6264f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:06.974816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eb439b11-7c34-4dea-a258-41c622e6264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:07.042248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 996/1998] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:07.052279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:07.054133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:07.112128 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ddb1e-1521-402d-bc50-1cccbd4cba66 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:07.117782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b2f68-cbbc-46dc-9d4a-9d8dc109ab5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:07.117782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b2f68-cbbc-46dc-9d4a-9d8dc109ab5d service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:07.117782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b2f68-cbbc-46dc-9d4a-9d8dc109ab5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:07.117782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b2f68-cbbc-46dc-9d4a-9d8dc109ab5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:07.130439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:07.133509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b2f68-cbbc-46dc-9d4a-9d8dc109ab5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:07.135533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b2f68-cbbc-46dc-9d4a-9d8dc109ab5d service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:07.147178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1003/1999] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:07.658614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb439b11-7c34-4dea-a258-41c622e6264f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:07.660150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eb439b11-7c34-4dea-a258-41c622e6264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:07.672313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb439b11-7c34-4dea-a258-41c622e6264f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:07.677183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:07.678054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:07.678337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:07.678616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] PATH_INFO: `/v3/projects/e85df18c80d54d0faf9c6cfd49586117` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:07.679802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e85df18c80d54d0faf9c6cfd49586117)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:07.681860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:07.849339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e85df18c80d54d0faf9c6cfd49586117', 'request_id': 'req-e56de38d-307a-43f6-afe8-9dfaf5429818'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:07.947918 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e85df18c80d54d0faf9c6cfd49586117', 'request_id': 'req-e56de38d-307a-43f6-afe8-9dfaf5429818'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:07.954778 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e85df18c80d54d0faf9c6cfd49586117', 'request_id': 'req-e56de38d-307a-43f6-afe8-9dfaf5429818'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:07.966417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] The token cache is being invalidate because project e85df18c80d54d0faf9c6cfd49586117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:07.966704 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:07.967352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:07.969243 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 997/2000] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:07 2022] DELETE /identity/v3/projects/e85df18c80d54d0faf9c6cfd49586117 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:07.973003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b2f68-cbbc-46dc-9d4a-9d8dc109ab5d service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:07.984461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b2f68-cbbc-46dc-9d4a-9d8dc109ab5d service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:08.044471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:08.045391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:08.045673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:08.045945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] PATH_INFO: `/v3/users/113b470153c944a282dd2104fc7c30a8` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:08.047148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=113b470153c944a282dd2104fc7c30a8)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:08.049083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:08.259313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113b470153c944a282dd2104fc7c30a8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:08.269232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113b470153c944a282dd2104fc7c30a8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:08.274845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113b470153c944a282dd2104fc7c30a8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:08.282464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113b470153c944a282dd2104fc7c30a8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:08.366630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:08.368281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:08.379167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113b470153c944a282dd2104fc7c30a8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:08.389884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1004/2001] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:07 2022] DELETE /identity/v3/users/113b470153c944a282dd2104fc7c30a8 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:08.442772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56de38d-307a-43f6-afe8-9dfaf5429818 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:08.455800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:08.525563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8097c46-89eb-45cc-9c91-b020bc576869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:08.526088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8097c46-89eb-45cc-9c91-b020bc576869 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:08.526369 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8097c46-89eb-45cc-9c91-b020bc576869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:08.526710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8097c46-89eb-45cc-9c91-b020bc576869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:08.528485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a8097c46-89eb-45cc-9c91-b020bc576869 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:08.602370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8097c46-89eb-45cc-9c91-b020bc576869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:08.604016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8097c46-89eb-45cc-9c91-b020bc576869 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:08.621637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 998/2002] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:45:08.891814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:08.893558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:08.993058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b74e5-6946-496f-ad6b-bdf04dd07b2e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:08.998414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:08.999315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:08.999600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:08.999894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] PATH_INFO: `/v3/projects/6f8dd37346fe459fb9f643505d53f746` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:09.001689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f8dd37346fe459fb9f643505d53f746)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:09.003901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:09.025466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:09.163671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f8dd37346fe459fb9f643505d53f746', 'request_id': 'req-527d199a-962f-4c40-a0ac-b5ee291a3c01'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:09.170114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f8dd37346fe459fb9f643505d53f746', 'request_id': 'req-527d199a-962f-4c40-a0ac-b5ee291a3c01'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:09.175653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f8dd37346fe459fb9f643505d53f746', 'request_id': 'req-527d199a-962f-4c40-a0ac-b5ee291a3c01'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:09.283319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] The token cache is being invalidate because project 6f8dd37346fe459fb9f643505d53f746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:09.283319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:09.285904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:09.287908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1005/2003] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:08 2022] DELETE /identity/v3/projects/6f8dd37346fe459fb9f643505d53f746 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:09.294491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8097c46-89eb-45cc-9c91-b020bc576869 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:09.312500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8097c46-89eb-45cc-9c91-b020bc576869 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:09.383877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:09.385490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:09.385783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:09.386066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] PATH_INFO: `/v3/users/0c69bda44d7d4842a4a3c4ddd7918249` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:09.388574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c69bda44d7d4842a4a3c4ddd7918249)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:09.391798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:09.819288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c69bda44d7d4842a4a3c4ddd7918249'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:09.829235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c69bda44d7d4842a4a3c4ddd7918249'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:09.841877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:09.845518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c69bda44d7d4842a4a3c4ddd7918249'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:09.959258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c69bda44d7d4842a4a3c4ddd7918249'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:09.960001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9eecd6ba-bac4-4a55-a654-9735f862c42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c69bda44d7d4842a4a3c4ddd7918249'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:09.970081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 999/2004] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:09 2022] DELETE /identity/v3/users/0c69bda44d7d4842a4a3c4ddd7918249 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:10.455524 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:10.456991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:10.482246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d199a-962f-4c40-a0ac-b5ee291a3c01 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:10.557502 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:10.558530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:10.559195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:10.559195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] PATH_INFO: `/v3/projects/6e078a3c3baa4c539ba3bbb2ee21de33` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:10.560427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e078a3c3baa4c539ba3bbb2ee21de33)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:10.562580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:10.718665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e078a3c3baa4c539ba3bbb2ee21de33', 'request_id': 'req-43037896-8d34-444a-ba71-e5fd85074051'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:10.725632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e078a3c3baa4c539ba3bbb2ee21de33', 'request_id': 'req-43037896-8d34-444a-ba71-e5fd85074051'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:10.747061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e078a3c3baa4c539ba3bbb2ee21de33', 'request_id': 'req-43037896-8d34-444a-ba71-e5fd85074051'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:10.852838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] The token cache is being invalidate because project 6e078a3c3baa4c539ba3bbb2ee21de33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:10.853124 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:10.853776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-43037896-8d34-444a-ba71-e5fd85074051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:10.855877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1006/2005] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:10 2022] DELETE /identity/v3/projects/6e078a3c3baa4c539ba3bbb2ee21de33 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:10.861388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff080ddd-591c-4158-b2b1-495b620e1747 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:10.861867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff080ddd-591c-4158-b2b1-495b620e1747 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:10.861951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff080ddd-591c-4158-b2b1-495b620e1747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:10.880881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ff080ddd-591c-4158-b2b1-495b620e1747 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:10.882120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ff080ddd-591c-4158-b2b1-495b620e1747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:10.944182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1000/2006] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:10.951736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e2eea-1794-4a92-8825-f6c641543ab2 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:10.952027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e2eea-1794-4a92-8825-f6c641543ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:10.952305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e2eea-1794-4a92-8825-f6c641543ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:10.973334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-229e2eea-1794-4a92-8825-f6c641543ab2 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:10.974561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-229e2eea-1794-4a92-8825-f6c641543ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:10.983732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1007/2007] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:10.988370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff080ddd-591c-4158-b2b1-495b620e1747 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:10.999015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff080ddd-591c-4158-b2b1-495b620e1747 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:11.003772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c0532-192b-4d38-bde2-84afe9ef852d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:11.004500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c0532-192b-4d38-bde2-84afe9ef852d admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:11.004781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c0532-192b-4d38-bde2-84afe9ef852d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:11.005051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c0532-192b-4d38-bde2-84afe9ef852d admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:11.005881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c0532-192b-4d38-bde2-84afe9ef852d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:45:11.006187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c0532-192b-4d38-bde2-84afe9ef852d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:11.007960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c0532-192b-4d38-bde2-84afe9ef852d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:11.020066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1001/2008] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:45:11.028763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e2eea-1794-4a92-8825-f6c641543ab2 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:11.038824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e2eea-1794-4a92-8825-f6c641543ab2 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:11.045858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb9ef4-9111-42f3-a539-2723fca208c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:11.046725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb9ef4-9111-42f3-a539-2723fca208c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:11.047225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb9ef4-9111-42f3-a539-2723fca208c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:11.047355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb9ef4-9111-42f3-a539-2723fca208c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:11.048282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb9ef4-9111-42f3-a539-2723fca208c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:11.050510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb9ef4-9111-42f3-a539-2723fca208c9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:11.176822 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1008/2009] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:11 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:11.181057 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c0532-192b-4d38-bde2-84afe9ef852d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:11.191312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c0532-192b-4d38-bde2-84afe9ef852d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:11.252205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4f977-8135-42ae-9862-66db83d78539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:11.252905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4f977-8135-42ae-9862-66db83d78539 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:11.253224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4f977-8135-42ae-9862-66db83d78539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:11.253517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4f977-8135-42ae-9862-66db83d78539 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:11.254234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4f977-8135-42ae-9862-66db83d78539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:11.256337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4f977-8135-42ae-9862-66db83d78539 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:11.337648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1002/2010] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:11 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:11.339049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb9ef4-9111-42f3-a539-2723fca208c9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:11.360427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb9ef4-9111-42f3-a539-2723fca208c9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:11.366183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefa9d4-92c2-4191-b50f-02680a33f0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:11.367013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefa9d4-92c2-4191-b50f-02680a33f0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:11.367436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefa9d4-92c2-4191-b50f-02680a33f0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:11.368092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefa9d4-92c2-4191-b50f-02680a33f0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:11.368675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefa9d4-92c2-4191-b50f-02680a33f0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:11.370465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefa9d4-92c2-4191-b50f-02680a33f0f1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:11.373342 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:11.397015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1009/2011] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:11.398366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4f977-8135-42ae-9862-66db83d78539 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:11.411619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4f977-8135-42ae-9862-66db83d78539 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:11.416601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013739e-1dfa-4c5d-a8cc-f6fe61ec8e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:11.417319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013739e-1dfa-4c5d-a8cc-f6fe61ec8e69 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:11.417629 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013739e-1dfa-4c5d-a8cc-f6fe61ec8e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:11.417947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013739e-1dfa-4c5d-a8cc-f6fe61ec8e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:11.418678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013739e-1dfa-4c5d-a8cc-f6fe61ec8e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:11.420299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013739e-1dfa-4c5d-a8cc-f6fe61ec8e69 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:11.434401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1003/2012] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:11.435863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefa9d4-92c2-4191-b50f-02680a33f0f1 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:11.448089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefa9d4-92c2-4191-b50f-02680a33f0f1 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:11.457756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dffad-95ec-4de6-b0c1-9929b9ffbcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:11.457756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dffad-95ec-4de6-b0c1-9929b9ffbcb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:11.457756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dffad-95ec-4de6-b0c1-9929b9ffbcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:11.457756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dffad-95ec-4de6-b0c1-9929b9ffbcb2 admin admin] PATH_INFO: `/v3/projects/3e42823305674a07acd1281462b6d68f/users/13f9ef55d1114892b67c812a74daca22/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:11.481704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dffad-95ec-4de6-b0c1-9929b9ffbcb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e42823305674a07acd1281462b6d68f, user_id=13f9ef55d1114892b67c812a74daca22, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:11.481704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dffad-95ec-4de6-b0c1-9929b9ffbcb2 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:11.607166 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1010/2013] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:11 2022] PUT /identity/v3/projects/3e42823305674a07acd1281462b6d68f/users/13f9ef55d1114892b67c812a74daca22/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:11.608948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09faae75-4965-4194-8549-40bd3d6a006b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:11.609272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09faae75-4965-4194-8549-40bd3d6a006b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:11.609550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09faae75-4965-4194-8549-40bd3d6a006b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:11.642326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-09faae75-4965-4194-8549-40bd3d6a006b None None] MFA Rules not processed for user `13f9ef55d1114892b67c812a74daca22`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:11.644119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-09faae75-4965-4194-8549-40bd3d6a006b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:11.695026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:11.718320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1004/2014] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:11.722852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dffad-95ec-4de6-b0c1-9929b9ffbcb2 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:11.785700 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dffad-95ec-4de6-b0c1-9929b9ffbcb2 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:11.791010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd158f-80f9-4eb3-868c-407a6fef5014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:11.791401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd158f-80f9-4eb3-868c-407a6fef5014 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:11.792713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd158f-80f9-4eb3-868c-407a6fef5014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:11.792713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd158f-80f9-4eb3-868c-407a6fef5014 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:11.792893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd158f-80f9-4eb3-868c-407a6fef5014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:11.795459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd158f-80f9-4eb3-868c-407a6fef5014 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:11.895331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1011/2015] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:11 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:45:11.899391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09faae75-4965-4194-8549-40bd3d6a006b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:11.911158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09faae75-4965-4194-8549-40bd3d6a006b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:11.980946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164a2d7-f047-4bcd-a2f9-8f30351ea79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:11.982031 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164a2d7-f047-4bcd-a2f9-8f30351ea79c admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:11.982490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164a2d7-f047-4bcd-a2f9-8f30351ea79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:11.983039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164a2d7-f047-4bcd-a2f9-8f30351ea79c admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:11.984046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164a2d7-f047-4bcd-a2f9-8f30351ea79c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:11.986191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164a2d7-f047-4bcd-a2f9-8f30351ea79c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:12.085288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1005/2016] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:11 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:12.089276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd158f-80f9-4eb3-868c-407a6fef5014 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:12.099868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd158f-80f9-4eb3-868c-407a6fef5014 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:12.104226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cb747-2738-4395-b608-424aef6f53f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:12.105466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cb747-2738-4395-b608-424aef6f53f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.105669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cb747-2738-4395-b608-424aef6f53f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.105946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cb747-2738-4395-b608-424aef6f53f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.106684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cb747-2738-4395-b608-424aef6f53f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:12.108294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cb747-2738-4395-b608-424aef6f53f3 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:12.119258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1012/2017] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:12.123823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164a2d7-f047-4bcd-a2f9-8f30351ea79c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:12.133807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164a2d7-f047-4bcd-a2f9-8f30351ea79c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:12.138586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcc1a8-7e28-449e-a1bd-6e83bb7fdd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:12.139253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcc1a8-7e28-449e-a1bd-6e83bb7fdd83 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.139541 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcc1a8-7e28-449e-a1bd-6e83bb7fdd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.139818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcc1a8-7e28-449e-a1bd-6e83bb7fdd83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.140552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcc1a8-7e28-449e-a1bd-6e83bb7fdd83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:12.142183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcc1a8-7e28-449e-a1bd-6e83bb7fdd83 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:12.152708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1006/2018] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:12.156972 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cb747-2738-4395-b608-424aef6f53f3 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:12.167061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cb747-2738-4395-b608-424aef6f53f3 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:12.172471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d3a20-3dc0-4fc6-ab5c-bf64c2f4f8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:12.179122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d3a20-3dc0-4fc6-ab5c-bf64c2f4f8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.179122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d3a20-3dc0-4fc6-ab5c-bf64c2f4f8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.179122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d3a20-3dc0-4fc6-ab5c-bf64c2f4f8e9 admin admin] PATH_INFO: `/v3/projects/fd1e2f000c534c1b8f2036a3b9bf107d/users/cb553cd529884668a0dde97744f8fef1/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.190645 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d3a20-3dc0-4fc6-ab5c-bf64c2f4f8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd1e2f000c534c1b8f2036a3b9bf107d, user_id=cb553cd529884668a0dde97744f8fef1, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:12.193007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17d3d1-dfeb-4f7d-a8f1-2df31b83a7a5 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.193234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d3a20-3dc0-4fc6-ab5c-bf64c2f4f8e9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:12.193411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17d3d1-dfeb-4f7d-a8f1-2df31b83a7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.193793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17d3d1-dfeb-4f7d-a8f1-2df31b83a7a5 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.195994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1007/2019] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:45:12.285796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1013/2020] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:12 2022] PUT /identity/v3/projects/fd1e2f000c534c1b8f2036a3b9bf107d/users/cb553cd529884668a0dde97744f8fef1/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:12.298091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909012a5-a9a9-4348-8426-69147b0a092e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.298823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909012a5-a9a9-4348-8426-69147b0a092e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.299329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909012a5-a9a9-4348-8426-69147b0a092e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.330936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-909012a5-a9a9-4348-8426-69147b0a092e None None] MFA Rules not processed for user `cb553cd529884668a0dde97744f8fef1`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:12.332233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-909012a5-a9a9-4348-8426-69147b0a092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:12.400264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1008/2021] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:12.405308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d3a20-3dc0-4fc6-ab5c-bf64c2f4f8e9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:12.407274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f04f2-5682-4279-8e4e-475577ddd376 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.408236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f04f2-5682-4279-8e4e-475577ddd376 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.408438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f04f2-5682-4279-8e4e-475577ddd376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.428994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cc1f04f2-5682-4279-8e4e-475577ddd376 None None] MFA Rules not processed for user `fa879d581c6d44e18a8703e66ffbd549`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:12.430487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cc1f04f2-5682-4279-8e4e-475577ddd376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:12.477693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d3a20-3dc0-4fc6-ab5c-bf64c2f4f8e9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:12.483312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3a79a-bce2-407a-b1d8-ad2ba565a180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:12.483974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3a79a-bce2-407a-b1d8-ad2ba565a180 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.484290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3a79a-bce2-407a-b1d8-ad2ba565a180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.484569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3a79a-bce2-407a-b1d8-ad2ba565a180 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.485303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3a79a-bce2-407a-b1d8-ad2ba565a180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:12.487219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3a79a-bce2-407a-b1d8-ad2ba565a180 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:12.494150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1009/2022] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:45:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:12.506001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f04f2-5682-4279-8e4e-475577ddd376 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:12.506234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cc1f04f2-5682-4279-8e4e-475577ddd376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:12.519271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f04f2-5682-4279-8e4e-475577ddd376 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:12.525522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d42f56-2aaa-47e8-912f-2394a98db7a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:12.526179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d42f56-2aaa-47e8-912f-2394a98db7a4 service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.526464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d42f56-2aaa-47e8-912f-2394a98db7a4 service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.527307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d42f56-2aaa-47e8-912f-2394a98db7a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.536516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d42f56-2aaa-47e8-912f-2394a98db7a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:12.538222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d42f56-2aaa-47e8-912f-2394a98db7a4 service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:12.549775 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1010/2023] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:45:12.563411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d42f56-2aaa-47e8-912f-2394a98db7a4 service placement] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:12.565090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a8d42f56-2aaa-47e8-912f-2394a98db7a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:12.580665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:12.583471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d42f56-2aaa-47e8-912f-2394a98db7a4 service placement] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:12.588119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:12.588987 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.590061 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.590345 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] PATH_INFO: `/v3/users/760c5416ab5c44a59eb03971c2252725` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.591549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=760c5416ab5c44a59eb03971c2252725)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:12.595018 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:12.638558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1014/2024] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:12 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:12.642857 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3a79a-bce2-407a-b1d8-ad2ba565a180 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:12.653462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:12.729594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3a79a-bce2-407a-b1d8-ad2ba565a180 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:12.736098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044a603-a74c-4487-8822-01b14642db7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:12.736936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044a603-a74c-4487-8822-01b14642db7e admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.737358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044a603-a74c-4487-8822-01b14642db7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.737731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044a603-a74c-4487-8822-01b14642db7e admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.738546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044a603-a74c-4487-8822-01b14642db7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:12.740573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044a603-a74c-4487-8822-01b14642db7e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:12.799038 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760c5416ab5c44a59eb03971c2252725'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:12.805866 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760c5416ab5c44a59eb03971c2252725'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:12.817808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760c5416ab5c44a59eb03971c2252725'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:12.833616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1015/2025] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:12 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:12.835777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044a603-a74c-4487-8822-01b14642db7e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:12.845426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044a603-a74c-4487-8822-01b14642db7e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:12.850463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeba9f7-41c4-4736-ab6e-2166c17dee04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:12.851191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeba9f7-41c4-4736-ab6e-2166c17dee04 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.851478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeba9f7-41c4-4736-ab6e-2166c17dee04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.851761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeba9f7-41c4-4736-ab6e-2166c17dee04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.852536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeba9f7-41c4-4736-ab6e-2166c17dee04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:12.854171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeba9f7-41c4-4736-ab6e-2166c17dee04 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:12.864625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1016/2026] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:45:12.868652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeba9f7-41c4-4736-ab6e-2166c17dee04 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:12.879285 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeba9f7-41c4-4736-ab6e-2166c17dee04 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:12.883890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae637b-9c14-4878-8ac7-e6383baa1803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:12.884764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae637b-9c14-4878-8ac7-e6383baa1803 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:12.885041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae637b-9c14-4878-8ac7-e6383baa1803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:12.885348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae637b-9c14-4878-8ac7-e6383baa1803 admin admin] PATH_INFO: `/v3/projects/ad8e1d9f98e349ac8004d00d4dbb23c3/users/bd18fc9e7ab2441180c17f1b87ba2068/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:12.893246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760c5416ab5c44a59eb03971c2252725'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:12.896823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760c5416ab5c44a59eb03971c2252725'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:12.901558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1011/2027] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:12 2022] DELETE /identity/v3/users/760c5416ab5c44a59eb03971c2252725 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:12.902160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae637b-9c14-4878-8ac7-e6383baa1803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8e1d9f98e349ac8004d00d4dbb23c3, user_id=bd18fc9e7ab2441180c17f1b87ba2068, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:12.904925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae637b-9c14-4878-8ac7-e6383baa1803 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:12.980916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1017/2028] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:12 2022] PUT /identity/v3/projects/ad8e1d9f98e349ac8004d00d4dbb23c3/users/bd18fc9e7ab2441180c17f1b87ba2068/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:12.984762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:12.986216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:13.056708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48626559-b2eb-46c9-b2ec-3789d076df09 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:13.103912 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:13.113197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03c086-9849-4028-a04f-712419efc77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:13.113874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03c086-9849-4028-a04f-712419efc77a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.114161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03c086-9849-4028-a04f-712419efc77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.114438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03c086-9849-4028-a04f-712419efc77a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.115263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03c086-9849-4028-a04f-712419efc77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:13.116861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03c086-9849-4028-a04f-712419efc77a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:13.127693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1012/2029] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:13.131989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae637b-9c14-4878-8ac7-e6383baa1803 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:13.142447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae637b-9c14-4878-8ac7-e6383baa1803 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:13.147182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350ae76-88fe-48a4-b7b1-471663db2270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:13.148083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ae76-88fe-48a4-b7b1-471663db2270 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.148366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ae76-88fe-48a4-b7b1-471663db2270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.148651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ae76-88fe-48a4-b7b1-471663db2270 admin admin] PATH_INFO: `/v3/projects/ad8e1d9f98e349ac8004d00d4dbb23c3/users/bd18fc9e7ab2441180c17f1b87ba2068/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.151073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350ae76-88fe-48a4-b7b1-471663db2270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8e1d9f98e349ac8004d00d4dbb23c3, user_id=bd18fc9e7ab2441180c17f1b87ba2068, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:13.153294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03c086-9849-4028-a04f-712419efc77a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:13.154426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350ae76-88fe-48a4-b7b1-471663db2270 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:13.155439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7c03c086-9849-4028-a04f-712419efc77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:13.166509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1018/2030] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:13 2022] PUT /identity/v3/projects/ad8e1d9f98e349ac8004d00d4dbb23c3/users/bd18fc9e7ab2441180c17f1b87ba2068/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:13.171746 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03c086-9849-4028-a04f-712419efc77a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:13.177860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e526de8-66b5-486f-b219-f8fb83177ed8 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.178142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e526de8-66b5-486f-b219-f8fb83177ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.178425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e526de8-66b5-486f-b219-f8fb83177ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.208251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-5e526de8-66b5-486f-b219-f8fb83177ed8 None None] MFA Rules not processed for user `bd18fc9e7ab2441180c17f1b87ba2068`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:13.209497 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5e526de8-66b5-486f-b219-f8fb83177ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:13.237834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:13.238860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.239154 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.239534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] PATH_INFO: `/v3/projects/5bb446eb04c04b73aacbcf5e605a6c76` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.240678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb446eb04c04b73aacbcf5e605a6c76)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:13.242795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:13.285136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1019/2031] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:13.297379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e526de8-66b5-486f-b219-f8fb83177ed8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:13.298758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5e526de8-66b5-486f-b219-f8fb83177ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:13.337757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:13.363536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e526de8-66b5-486f-b219-f8fb83177ed8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:13.376235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb446eb04c04b73aacbcf5e605a6c76', 'request_id': 'req-6e2f985a-2324-44ce-ab68-60bd4f45be06'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:13.418002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffd231-2104-444d-a9e0-4b3fe0af338c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:13.418566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffd231-2104-444d-a9e0-4b3fe0af338c service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.418847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffd231-2104-444d-a9e0-4b3fe0af338c service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.419123 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffd231-2104-444d-a9e0-4b3fe0af338c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.420985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5bffd231-2104-444d-a9e0-4b3fe0af338c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:13.476350 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffd231-2104-444d-a9e0-4b3fe0af338c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:13.477885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffd231-2104-444d-a9e0-4b3fe0af338c service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:13.487224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1020/2032] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:45:13.488000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffd231-2104-444d-a9e0-4b3fe0af338c service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:13.489351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5bffd231-2104-444d-a9e0-4b3fe0af338c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:13.500927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb446eb04c04b73aacbcf5e605a6c76', 'request_id': 'req-6e2f985a-2324-44ce-ab68-60bd4f45be06'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:13.502088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb446eb04c04b73aacbcf5e605a6c76', 'request_id': 'req-6e2f985a-2324-44ce-ab68-60bd4f45be06'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:13.517084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] The token cache is being invalidate because project 5bb446eb04c04b73aacbcf5e605a6c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:13.517678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:13.518676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-6e2f985a-2324-44ce-ab68-60bd4f45be06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:13.524683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1013/2033] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:13 2022] DELETE /identity/v3/projects/5bb446eb04c04b73aacbcf5e605a6c76 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:13.527900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd42d8-6028-4e73-b80f-a323ae7273e2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.528302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd42d8-6028-4e73-b80f-a323ae7273e2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.528581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd42d8-6028-4e73-b80f-a323ae7273e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.553044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0bcd42d8-6028-4e73-b80f-a323ae7273e2 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:13.554624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0bcd42d8-6028-4e73-b80f-a323ae7273e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:13.561774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffd231-2104-444d-a9e0-4b3fe0af338c service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:13.564716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1014/2034] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:13.571710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:13.572659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.572789 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953cd13-f3c9-4ddb-9a46-239052f7d999 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.577302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.577417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953cd13-f3c9-4ddb-9a46-239052f7d999 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.577511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] PATH_INFO: `/v3/users/6b86eed4b4394339bc11072444152c8b` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.577601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953cd13-f3c9-4ddb-9a46-239052f7d999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.577686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b86eed4b4394339bc11072444152c8b)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:13.577686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:13.599495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9953cd13-f3c9-4ddb-9a46-239052f7d999 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:13.601298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9953cd13-f3c9-4ddb-9a46-239052f7d999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:13.613583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1015/2035] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:45:13.617474 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953cd13-f3c9-4ddb-9a46-239052f7d999 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:13.627579 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953cd13-f3c9-4ddb-9a46-239052f7d999 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:13.633268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ee1a4-2fa6-433d-af7e-e20024911c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:13.633947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ee1a4-2fa6-433d-af7e-e20024911c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.634219 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ee1a4-2fa6-433d-af7e-e20024911c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.634537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ee1a4-2fa6-433d-af7e-e20024911c4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.635256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ee1a4-2fa6-433d-af7e-e20024911c4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:45:13.635560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ee1a4-2fa6-433d-af7e-e20024911c4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:13.637799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ee1a4-2fa6-433d-af7e-e20024911c4f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:13.650151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1016/2036] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:45:13.655073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ee1a4-2fa6-433d-af7e-e20024911c4f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:13.664861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ee1a4-2fa6-433d-af7e-e20024911c4f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:13.669662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ea84e-4aaa-479f-8b7d-f75290037173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:13.670244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ea84e-4aaa-479f-8b7d-f75290037173 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.670527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ea84e-4aaa-479f-8b7d-f75290037173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.670799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ea84e-4aaa-479f-8b7d-f75290037173 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.671479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ea84e-4aaa-479f-8b7d-f75290037173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:13.673396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ea84e-4aaa-479f-8b7d-f75290037173 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:13.764149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1017/2037] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:13 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:45:13.765543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ea84e-4aaa-479f-8b7d-f75290037173 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:13.770670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b86eed4b4394339bc11072444152c8b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:13.771569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b86eed4b4394339bc11072444152c8b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:13.786548 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b86eed4b4394339bc11072444152c8b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:13.793819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b86eed4b4394339bc11072444152c8b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:13.831971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ea84e-4aaa-479f-8b7d-f75290037173 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:13.836805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5fffc-1555-4474-923b-1fb57ca3b2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:13.837494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5fffc-1555-4474-923b-1fb57ca3b2ed admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.837787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5fffc-1555-4474-923b-1fb57ca3b2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.838064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5fffc-1555-4474-923b-1fb57ca3b2ed admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.838757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5fffc-1555-4474-923b-1fb57ca3b2ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:13.841376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5fffc-1555-4474-923b-1fb57ca3b2ed admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:13.871279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:13.916343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-df21c38d-6e7f-40ce-9f9a-13d7238e2cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b86eed4b4394339bc11072444152c8b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:13.937561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1021/2038] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:13 2022] DELETE /identity/v3/users/6b86eed4b4394339bc11072444152c8b => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:13.937561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4347d88-c4ba-47f0-8a26-8b0c4c82ccf5 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:13.937561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4347d88-c4ba-47f0-8a26-8b0c4c82ccf5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:13.937561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4347d88-c4ba-47f0-8a26-8b0c4c82ccf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:13.945766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-f4347d88-c4ba-47f0-8a26-8b0c4c82ccf5 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:13.946967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f4347d88-c4ba-47f0-8a26-8b0c4c82ccf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:13.957931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:13.981750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1018/2039] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:13 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:13.984740 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5fffc-1555-4474-923b-1fb57ca3b2ed admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:13.986477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fde5fffc-1555-4474-923b-1fb57ca3b2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:13.999148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5fffc-1555-4474-923b-1fb57ca3b2ed admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:14.012219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1022/2040] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:45:14.023163 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4347d88-c4ba-47f0-8a26-8b0c4c82ccf5 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:14.024848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f4347d88-c4ba-47f0-8a26-8b0c4c82ccf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:14.057291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a450ee-2215-4867-8a46-7d9620125480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:14.059862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a450ee-2215-4867-8a46-7d9620125480 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:14.059862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a450ee-2215-4867-8a46-7d9620125480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:14.059862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a450ee-2215-4867-8a46-7d9620125480 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:14.059862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a450ee-2215-4867-8a46-7d9620125480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:14.061557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a450ee-2215-4867-8a46-7d9620125480 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:14.071697 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1019/2041] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:45:14.076076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a450ee-2215-4867-8a46-7d9620125480 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:14.083245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4347d88-c4ba-47f0-8a26-8b0c4c82ccf5 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:14.086183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a450ee-2215-4867-8a46-7d9620125480 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:14.087103 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a124c8-9ba7-420e-94c9-6b7c58b2841d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:14.087767 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a124c8-9ba7-420e-94c9-6b7c58b2841d service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:14.088050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a124c8-9ba7-420e-94c9-6b7c58b2841d service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:14.088330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a124c8-9ba7-420e-94c9-6b7c58b2841d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:14.091971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2be71-789e-47ed-9959-afb916998cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:14.092627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2be71-789e-47ed-9959-afb916998cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:14.092909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2be71-789e-47ed-9959-afb916998cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:14.093240 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2be71-789e-47ed-9959-afb916998cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:14.093997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2be71-789e-47ed-9959-afb916998cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:14.099945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2be71-789e-47ed-9959-afb916998cb0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:14.100057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a124c8-9ba7-420e-94c9-6b7c58b2841d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:14.100057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a124c8-9ba7-420e-94c9-6b7c58b2841d service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:14.106495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1020/2042] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:45:14.109262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1023/2043] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:45:14.110569 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2be71-789e-47ed-9959-afb916998cb0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:14.121192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2be71-789e-47ed-9959-afb916998cb0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:14.126321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b8db4-90ea-4cee-bf42-1283243c5509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:14.127343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b8db4-90ea-4cee-bf42-1283243c5509 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:14.134396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b8db4-90ea-4cee-bf42-1283243c5509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:14.134396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b8db4-90ea-4cee-bf42-1283243c5509 admin admin] PATH_INFO: `/v3/projects/951609a482e34e93908057a68c5dc706/users/7bf3f705e04640a2a5d9bedb71c6ccb5/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:14.144140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b8db4-90ea-4cee-bf42-1283243c5509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=951609a482e34e93908057a68c5dc706, user_id=7bf3f705e04640a2a5d9bedb71c6ccb5, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:14.146886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b8db4-90ea-4cee-bf42-1283243c5509 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:14.248874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1021/2044] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:14 2022] PUT /identity/v3/projects/951609a482e34e93908057a68c5dc706/users/7bf3f705e04640a2a5d9bedb71c6ccb5/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:14.258993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79d613-3f2e-4780-8a12-82463324330f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:14.259285 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79d613-3f2e-4780-8a12-82463324330f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:14.259571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79d613-3f2e-4780-8a12-82463324330f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:14.279559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ab79d613-3f2e-4780-8a12-82463324330f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:14.280865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ab79d613-3f2e-4780-8a12-82463324330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:14.363225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1024/2045] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:14.374180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b8db4-90ea-4cee-bf42-1283243c5509 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:14.432130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b8db4-90ea-4cee-bf42-1283243c5509 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:14.436799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9bb9-76f3-451d-ad58-e464ecfe18c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:14.437623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9bb9-76f3-451d-ad58-e464ecfe18c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:14.437914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9bb9-76f3-451d-ad58-e464ecfe18c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:14.438188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9bb9-76f3-451d-ad58-e464ecfe18c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:14.447766 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d9bb9-76f3-451d-ad58-e464ecfe18c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:14.449210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d9bb9-76f3-451d-ad58-e464ecfe18c6 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:14.464376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1022/2046] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:14.533038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79d613-3f2e-4780-8a12-82463324330f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:14.534732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ab79d613-3f2e-4780-8a12-82463324330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:14.553841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79d613-3f2e-4780-8a12-82463324330f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:14.564171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:14.565195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:14.565778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:14.566278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] PATH_INFO: `/v3/projects/ece2edfd519a4fe78a221681f87c2a62` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:14.568401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ece2edfd519a4fe78a221681f87c2a62)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:14.572345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:14.575252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:14.688098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ece2edfd519a4fe78a221681f87c2a62', 'request_id': 'req-3de73610-9146-42e9-b27d-39a0cb472949'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:14.694217 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ece2edfd519a4fe78a221681f87c2a62', 'request_id': 'req-3de73610-9146-42e9-b27d-39a0cb472949'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:14.699698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ece2edfd519a4fe78a221681f87c2a62', 'request_id': 'req-3de73610-9146-42e9-b27d-39a0cb472949'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:14.788389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] The token cache is being invalidate because project ece2edfd519a4fe78a221681f87c2a62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:14.788682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:14.789372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:14.790986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1025/2047] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:14 2022] DELETE /identity/v3/projects/ece2edfd519a4fe78a221681f87c2a62 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:14.795692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9bb9-76f3-451d-ad58-e464ecfe18c6 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:14.806633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9bb9-76f3-451d-ad58-e464ecfe18c6 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:14.843303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:14.889144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:14.890049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:14.890336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:14.890611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] PATH_INFO: `/v3/users/3c162fae2a0d4e8c81a2808e4f811fc8` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:14.891812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c162fae2a0d4e8c81a2808e4f811fc8)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:14.893774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:15.164687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c162fae2a0d4e8c81a2808e4f811fc8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:15.171404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c162fae2a0d4e8c81a2808e4f811fc8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:15.182823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c162fae2a0d4e8c81a2808e4f811fc8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:15.322896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c162fae2a0d4e8c81a2808e4f811fc8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:15.323518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c162fae2a0d4e8c81a2808e4f811fc8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:15.335410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1023/2048] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:14 2022] DELETE /identity/v3/users/3c162fae2a0d4e8c81a2808e4f811fc8 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:15.567744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:15.569237 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:15.631565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de73610-9146-42e9-b27d-39a0cb472949 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:15.686868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec44f2-92f1-4c13-a187-897f87d1b48e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:15.687687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec44f2-92f1-4c13-a187-897f87d1b48e service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:15.688074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec44f2-92f1-4c13-a187-897f87d1b48e service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:15.688475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec44f2-92f1-4c13-a187-897f87d1b48e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:15.690410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-16ec44f2-92f1-4c13-a187-897f87d1b48e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:15.753112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec44f2-92f1-4c13-a187-897f87d1b48e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:15.754547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec44f2-92f1-4c13-a187-897f87d1b48e service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:15.759416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:15.764522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1026/2049] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:15.856946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:15.858491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:15.946339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6f538-9d60-40c5-8949-8d29351c4392 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:15.953524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:15.954600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:15.954889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:15.955168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] PATH_INFO: `/v3/projects/8736e59ad3aa4de5875955ae82d06917` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:15.957032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8736e59ad3aa4de5875955ae82d06917)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:15.959302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:16.111540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:16.114230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8736e59ad3aa4de5875955ae82d06917', 'request_id': 'req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:16.257060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8736e59ad3aa4de5875955ae82d06917', 'request_id': 'req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:16.265086 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8736e59ad3aa4de5875955ae82d06917', 'request_id': 'req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:16.279005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] The token cache is being invalidate because project 8736e59ad3aa4de5875955ae82d06917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:16.279292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:16.279964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:16.282052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1024/2050] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:15 2022] DELETE /identity/v3/projects/8736e59ad3aa4de5875955ae82d06917 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:16.285659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec44f2-92f1-4c13-a187-897f87d1b48e service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:16.299579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec44f2-92f1-4c13-a187-897f87d1b48e service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:16.360367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:16.361310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:16.363236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:16.363236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] PATH_INFO: `/v3/users/73dc177b67824d1d9e3095dcb2793687` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:16.363236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73dc177b67824d1d9e3095dcb2793687)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:16.364953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:16.597583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73dc177b67824d1d9e3095dcb2793687'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:16.598641 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73dc177b67824d1d9e3095dcb2793687'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:16.619754 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73dc177b67824d1d9e3095dcb2793687'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:16.624724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73dc177b67824d1d9e3095dcb2793687'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:16.702828 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73dc177b67824d1d9e3095dcb2793687'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:16.710071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1027/2051] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:16 2022] DELETE /identity/v3/users/73dc177b67824d1d9e3095dcb2793687 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:17.320959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:17.327578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:17.345694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573b40c-71fa-4e3e-8c13-f6dbde0f9d98 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:17.413617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:17.414603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:17.414949 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:17.415282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] PATH_INFO: `/v3/projects/9d80da6629424d66b3cff407b10d9836` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:17.416542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d80da6629424d66b3cff407b10d9836)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:17.418503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:17.528097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d80da6629424d66b3cff407b10d9836', 'request_id': 'req-c7d72bd9-f4d0-486a-be17-9e9ac0616541'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:17.613997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d80da6629424d66b3cff407b10d9836', 'request_id': 'req-c7d72bd9-f4d0-486a-be17-9e9ac0616541'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:17.620623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d80da6629424d66b3cff407b10d9836', 'request_id': 'req-c7d72bd9-f4d0-486a-be17-9e9ac0616541'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:17.621291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:17.627665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:17.633419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] The token cache is being invalidate because project 9d80da6629424d66b3cff407b10d9836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:17.633708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:17.634374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c7d72bd9-f4d0-486a-be17-9e9ac0616541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:17.636942 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1025/2052] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:17 2022] DELETE /identity/v3/projects/9d80da6629424d66b3cff407b10d9836 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:17.642895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc88d4-b674-4c46-81c5-c97f8192912e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:17.643535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc88d4-b674-4c46-81c5-c97f8192912e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:17.643958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc88d4-b674-4c46-81c5-c97f8192912e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:17.666152 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6bdc88d4-b674-4c46-81c5-c97f8192912e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:17.668785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6bdc88d4-b674-4c46-81c5-c97f8192912e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:17.696498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449752f-1fc8-41f0-a94e-5aed111eed84 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:17.746230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:17.767394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912474c8-a772-4c54-a303-18f5c0ffdf06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:17.768272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912474c8-a772-4c54-a303-18f5c0ffdf06 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:17.768614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912474c8-a772-4c54-a303-18f5c0ffdf06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:17.768944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912474c8-a772-4c54-a303-18f5c0ffdf06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:17.770554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1026/2053] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:17.772333 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-912474c8-a772-4c54-a303-18f5c0ffdf06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:17.779826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43cb7b-f26c-497c-8891-e03d9aaab6d4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:17.780302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43cb7b-f26c-497c-8891-e03d9aaab6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:17.780725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43cb7b-f26c-497c-8891-e03d9aaab6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:17.802663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ac43cb7b-f26c-497c-8891-e03d9aaab6d4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:17.804753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ac43cb7b-f26c-497c-8891-e03d9aaab6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:17.816322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1027/2054] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:17.819795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:17.822208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43cb7b-f26c-497c-8891-e03d9aaab6d4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:17.833855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43cb7b-f26c-497c-8891-e03d9aaab6d4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:17.840638 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231242e-ef45-4726-8c7c-f1aeb6b379de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:17.842025 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231242e-ef45-4726-8c7c-f1aeb6b379de admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:17.842308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231242e-ef45-4726-8c7c-f1aeb6b379de admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:17.842579 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231242e-ef45-4726-8c7c-f1aeb6b379de admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:17.843533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231242e-ef45-4726-8c7c-f1aeb6b379de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:45:17.843841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231242e-ef45-4726-8c7c-f1aeb6b379de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:17.846070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231242e-ef45-4726-8c7c-f1aeb6b379de admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:17.859503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912474c8-a772-4c54-a303-18f5c0ffdf06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:17.861303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912474c8-a772-4c54-a303-18f5c0ffdf06 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:17.867352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1028/2055] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:45:17.868284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231242e-ef45-4726-8c7c-f1aeb6b379de admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:17.877566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1028/2056] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:45:17.888459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231242e-ef45-4726-8c7c-f1aeb6b379de admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:17.893615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99e48d-9ec6-44ab-baae-f7c07772fb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:17.894253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99e48d-9ec6-44ab-baae-f7c07772fb61 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:17.894541 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99e48d-9ec6-44ab-baae-f7c07772fb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:17.894814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99e48d-9ec6-44ab-baae-f7c07772fb61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:17.895514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99e48d-9ec6-44ab-baae-f7c07772fb61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:17.897796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99e48d-9ec6-44ab-baae-f7c07772fb61 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:17.983764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1029/2057] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:17 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:17.988176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912474c8-a772-4c54-a303-18f5c0ffdf06 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:18.001620 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912474c8-a772-4c54-a303-18f5c0ffdf06 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:18.089021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244691a-1fa7-47c3-9d2b-0dddaf855385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:18.089799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244691a-1fa7-47c3-9d2b-0dddaf855385 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:18.090088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244691a-1fa7-47c3-9d2b-0dddaf855385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:18.090643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244691a-1fa7-47c3-9d2b-0dddaf855385 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:18.091763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244691a-1fa7-47c3-9d2b-0dddaf855385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:18.093773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244691a-1fa7-47c3-9d2b-0dddaf855385 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:18.224676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1029/2058] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:17 2022] POST /identity/v3/users => generated 288 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:18.225531 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99e48d-9ec6-44ab-baae-f7c07772fb61 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:18.239626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99e48d-9ec6-44ab-baae-f7c07772fb61 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:18.244869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d7d9b-808e-4045-804c-4c2d7271dd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:18.247013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d7d9b-808e-4045-804c-4c2d7271dd1e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:18.248889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d7d9b-808e-4045-804c-4c2d7271dd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:18.249270 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d7d9b-808e-4045-804c-4c2d7271dd1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:18.250118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d7d9b-808e-4045-804c-4c2d7271dd1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:18.251894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d7d9b-808e-4045-804c-4c2d7271dd1e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:18.264023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1030/2059] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:18.268148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244691a-1fa7-47c3-9d2b-0dddaf855385 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:18.283267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244691a-1fa7-47c3-9d2b-0dddaf855385 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:18.289006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbff8b-7b28-49d7-b4ae-395663b359ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:18.289993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbff8b-7b28-49d7-b4ae-395663b359ee admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:18.290498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbff8b-7b28-49d7-b4ae-395663b359ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:18.290948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbff8b-7b28-49d7-b4ae-395663b359ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:18.293039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbff8b-7b28-49d7-b4ae-395663b359ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:18.295599 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbff8b-7b28-49d7-b4ae-395663b359ee admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:18.311005 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1030/2060] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:18.314203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d7d9b-808e-4045-804c-4c2d7271dd1e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:18.326083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d7d9b-808e-4045-804c-4c2d7271dd1e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:18.332182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3203ea-7136-43e8-aa37-9dfbbe483ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:18.338272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3203ea-7136-43e8-aa37-9dfbbe483ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:18.338272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3203ea-7136-43e8-aa37-9dfbbe483ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:18.338272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3203ea-7136-43e8-aa37-9dfbbe483ba3 admin admin] PATH_INFO: `/v3/projects/314260256dbb4a8e8e2b4c1c5f857dcb/users/e5d9eb7a0f3f4195a8c2e68edeb21d6a/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:18.349611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3203ea-7136-43e8-aa37-9dfbbe483ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=314260256dbb4a8e8e2b4c1c5f857dcb, user_id=e5d9eb7a0f3f4195a8c2e68edeb21d6a, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:18.352293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3203ea-7136-43e8-aa37-9dfbbe483ba3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:18.462488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1031/2061] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:18 2022] PUT /identity/v3/projects/314260256dbb4a8e8e2b4c1c5f857dcb/users/e5d9eb7a0f3f4195a8c2e68edeb21d6a/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:18.484033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9bb60-f43e-4af3-adfa-107454f9b6ad None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:18.484592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9bb60-f43e-4af3-adfa-107454f9b6ad None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:18.485047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9bb60-f43e-4af3-adfa-107454f9b6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:18.515859 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9fc9bb60-f43e-4af3-adfa-107454f9b6ad None None] MFA Rules not processed for user `e5d9eb7a0f3f4195a8c2e68edeb21d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:18.517310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9fc9bb60-f43e-4af3-adfa-107454f9b6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:18.648718 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1031/2062] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:18.654441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3203ea-7136-43e8-aa37-9dfbbe483ba3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:18.731729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3203ea-7136-43e8-aa37-9dfbbe483ba3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:18.736742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2d8a3-cc2f-4009-832c-85d59cdeb65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:18.738560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2d8a3-cc2f-4009-832c-85d59cdeb65a admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:18.738858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2d8a3-cc2f-4009-832c-85d59cdeb65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:18.739135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2d8a3-cc2f-4009-832c-85d59cdeb65a admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:18.739846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2d8a3-cc2f-4009-832c-85d59cdeb65a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:18.742103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2d8a3-cc2f-4009-832c-85d59cdeb65a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:18.834064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1032/2063] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:18 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:45:18.838929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9bb60-f43e-4af3-adfa-107454f9b6ad None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:18.851046 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9bb60-f43e-4af3-adfa-107454f9b6ad None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:18.934269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54312-349c-4f5c-9214-40a844113c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:18.934772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54312-349c-4f5c-9214-40a844113c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:18.935293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54312-349c-4f5c-9214-40a844113c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:18.935666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54312-349c-4f5c-9214-40a844113c8a admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:18.936467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe54312-349c-4f5c-9214-40a844113c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:18.938553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe54312-349c-4f5c-9214-40a844113c8a admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:19.015298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1032/2064] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:18 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:19.019267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2d8a3-cc2f-4009-832c-85d59cdeb65a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:19.032130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2d8a3-cc2f-4009-832c-85d59cdeb65a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:19.037258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d1430-5f58-4547-9830-9b1e47ca6aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:19.038108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1430-5f58-4547-9830-9b1e47ca6aef admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:19.038537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1430-5f58-4547-9830-9b1e47ca6aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:19.038956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1430-5f58-4547-9830-9b1e47ca6aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:19.039857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1430-5f58-4547-9830-9b1e47ca6aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:19.041791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1430-5f58-4547-9830-9b1e47ca6aef admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:19.053611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1033/2065] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:19.058383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54312-349c-4f5c-9214-40a844113c8a admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:19.068513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54312-349c-4f5c-9214-40a844113c8a admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:19.073363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe28c2-98da-4d8c-a3f9-09815a667631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:19.074919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe28c2-98da-4d8c-a3f9-09815a667631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:19.075197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe28c2-98da-4d8c-a3f9-09815a667631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:19.075466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe28c2-98da-4d8c-a3f9-09815a667631 admin admin] PATH_INFO: `/v3/projects/04d1a61e44984ec4a76c6c099dc4a72f/users/29fcd54096ab4e688a48a5e857fbfb89/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:19.095979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe28c2-98da-4d8c-a3f9-09815a667631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d1a61e44984ec4a76c6c099dc4a72f, user_id=29fcd54096ab4e688a48a5e857fbfb89, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:19.099662 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe28c2-98da-4d8c-a3f9-09815a667631 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:19.192313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1033/2066] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:19 2022] PUT /identity/v3/projects/04d1a61e44984ec4a76c6c099dc4a72f/users/29fcd54096ab4e688a48a5e857fbfb89/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:19.196167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d1430-5f58-4547-9830-9b1e47ca6aef admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:19.208206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d1430-5f58-4547-9830-9b1e47ca6aef admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:19.275492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990c902-67e7-4810-b859-caf8a60f039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:19.276474 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990c902-67e7-4810-b859-caf8a60f039e admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:19.276796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990c902-67e7-4810-b859-caf8a60f039e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:19.277071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990c902-67e7-4810-b859-caf8a60f039e admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:19.278003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990c902-67e7-4810-b859-caf8a60f039e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:19.279600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990c902-67e7-4810-b859-caf8a60f039e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:19.295099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1034/2067] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:19.296006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe28c2-98da-4d8c-a3f9-09815a667631 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:19.306925 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe28c2-98da-4d8c-a3f9-09815a667631 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:19.312956 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc74a56-aac6-4dbf-8a62-99233fb3730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:19.314071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc74a56-aac6-4dbf-8a62-99233fb3730f admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:19.314434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc74a56-aac6-4dbf-8a62-99233fb3730f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:19.314791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc74a56-aac6-4dbf-8a62-99233fb3730f admin admin] PATH_INFO: `/v3/projects/04d1a61e44984ec4a76c6c099dc4a72f/users/29fcd54096ab4e688a48a5e857fbfb89/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:19.317120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc74a56-aac6-4dbf-8a62-99233fb3730f admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d1a61e44984ec4a76c6c099dc4a72f, user_id=29fcd54096ab4e688a48a5e857fbfb89, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:19.320066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc74a56-aac6-4dbf-8a62-99233fb3730f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:19.332912 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1034/2068] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:19 2022] PUT /identity/v3/projects/04d1a61e44984ec4a76c6c099dc4a72f/users/29fcd54096ab4e688a48a5e857fbfb89/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:19.345529 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba287cc-c7ec-4354-8f4a-bd14e812ccf7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:19.346095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba287cc-c7ec-4354-8f4a-bd14e812ccf7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:19.346095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba287cc-c7ec-4354-8f4a-bd14e812ccf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:19.347581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:19.378106 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0ba287cc-c7ec-4354-8f4a-bd14e812ccf7 None None] MFA Rules not processed for user `29fcd54096ab4e688a48a5e857fbfb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:19.379375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0ba287cc-c7ec-4354-8f4a-bd14e812ccf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:19.454845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1035/2069] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:19.466552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc74a56-aac6-4dbf-8a62-99233fb3730f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:19.468534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5fc74a56-aac6-4dbf-8a62-99233fb3730f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:19.497515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:19.533545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc74a56-aac6-4dbf-8a62-99233fb3730f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:19.537493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e732d7-a2ad-4af5-9c95-09da30a9b1c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:19.538144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e732d7-a2ad-4af5-9c95-09da30a9b1c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:19.538424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e732d7-a2ad-4af5-9c95-09da30a9b1c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:19.538696 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e732d7-a2ad-4af5-9c95-09da30a9b1c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:19.608725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e732d7-a2ad-4af5-9c95-09da30a9b1c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:19.610295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e732d7-a2ad-4af5-9c95-09da30a9b1c8 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:19.621361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1035/2070] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:20.491306 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:20.703320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46150-d08c-4d50-a60a-c2c07c05f138 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:20.703844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46150-d08c-4d50-a60a-c2c07c05f138 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:20.704153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46150-d08c-4d50-a60a-c2c07c05f138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:20.734670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fcf46150-d08c-4d50-a60a-c2c07c05f138 None None] MFA Rules not processed for user `7bf3f705e04640a2a5d9bedb71c6ccb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:20.736006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fcf46150-d08c-4d50-a60a-c2c07c05f138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:20.820431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1036/2071] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:20.831341 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef14df-8eec-4071-8e85-3fb3d2b0ae65 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:20.831341 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef14df-8eec-4071-8e85-3fb3d2b0ae65 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:20.831666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef14df-8eec-4071-8e85-3fb3d2b0ae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:20.853021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ecef14df-8eec-4071-8e85-3fb3d2b0ae65 None None] MFA Rules not processed for user `7bf3f705e04640a2a5d9bedb71c6ccb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:20.854319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ecef14df-8eec-4071-8e85-3fb3d2b0ae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:20.867829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1036/2072] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:20.878844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf46150-d08c-4d50-a60a-c2c07c05f138 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:20.880202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fcf46150-d08c-4d50-a60a-c2c07c05f138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:20.974072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf46150-d08c-4d50-a60a-c2c07c05f138 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:20.979847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b9afe-3db8-41e4-8b2e-c4856ff036f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:20.980493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b9afe-3db8-41e4-8b2e-c4856ff036f3 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:20.980736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b9afe-3db8-41e4-8b2e-c4856ff036f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:20.983305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b9afe-3db8-41e4-8b2e-c4856ff036f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:20.993596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b9afe-3db8-41e4-8b2e-c4856ff036f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:20.995279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b9afe-3db8-41e4-8b2e-c4856ff036f3 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:21.000278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:21.008760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1037/2073] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:21.832883 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0e720-2649-4bf4-837e-31acd8a3f669 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:21.833367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0e720-2649-4bf4-837e-31acd8a3f669 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:21.833477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0e720-2649-4bf4-837e-31acd8a3f669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:21.858066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9bb0e720-2649-4bf4-837e-31acd8a3f669 None None] MFA Rules not processed for user `7bf3f705e04640a2a5d9bedb71c6ccb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:21.871796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9bb0e720-2649-4bf4-837e-31acd8a3f669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:21.885006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1037/2074] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:22.908063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b9afe-3db8-41e4-8b2e-c4856ff036f3 service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:22.992801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b9afe-3db8-41e4-8b2e-c4856ff036f3 service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:22.998436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:22.999380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:22.999664 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:22.999926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] PATH_INFO: `/v3/users/e5d9eb7a0f3f4195a8c2e68edeb21d6a` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:23.001449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d9eb7a0f3f4195a8c2e68edeb21d6a)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:23.004505 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:23.249860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d9eb7a0f3f4195a8c2e68edeb21d6a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:23.268845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d9eb7a0f3f4195a8c2e68edeb21d6a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:23.298118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d9eb7a0f3f4195a8c2e68edeb21d6a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:23.408882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d9eb7a0f3f4195a8c2e68edeb21d6a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:23.409700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d9eb7a0f3f4195a8c2e68edeb21d6a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:23.420787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1038/2075] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:22 2022] DELETE /identity/v3/users/e5d9eb7a0f3f4195a8c2e68edeb21d6a => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:23.424235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375399b-0743-4da5-aa65-e2f5c2f5c4ae None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:23.424714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375399b-0743-4da5-aa65-e2f5c2f5c4ae None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:23.426757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375399b-0743-4da5-aa65-e2f5c2f5c4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:23.461315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-b375399b-0743-4da5-aa65-e2f5c2f5c4ae None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:23.461633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b375399b-0743-4da5-aa65-e2f5c2f5c4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:23.536730 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1038/2076] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:23.547908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:23.549838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:23.669277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33cbdf-a7de-4449-9d90-d51332c2a0dc admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:23.675074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6763195-8a6d-4dd1-95fe-6ef7ff604940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:23.675074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6763195-8a6d-4dd1-95fe-6ef7ff604940 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:23.675074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6763195-8a6d-4dd1-95fe-6ef7ff604940 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:23.675540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6763195-8a6d-4dd1-95fe-6ef7ff604940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:23.707681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6763195-8a6d-4dd1-95fe-6ef7ff604940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:23.710258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6763195-8a6d-4dd1-95fe-6ef7ff604940 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:23.732803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1039/2077] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:24.224277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375399b-0743-4da5-aa65-e2f5c2f5c4ae None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:24.224277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b375399b-0743-4da5-aa65-e2f5c2f5c4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:24.246363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375399b-0743-4da5-aa65-e2f5c2f5c4ae None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:24.252393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:24.253402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:24.253694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:24.253968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] PATH_INFO: `/v3/projects/314260256dbb4a8e8e2b4c1c5f857dcb` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:24.255497 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=314260256dbb4a8e8e2b4c1c5f857dcb)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:24.266161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:24.441520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '314260256dbb4a8e8e2b4c1c5f857dcb', 'request_id': 'req-2e4a1ae9-2767-42f3-8104-f409d369a2b2'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:24.449472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '314260256dbb4a8e8e2b4c1c5f857dcb', 'request_id': 'req-2e4a1ae9-2767-42f3-8104-f409d369a2b2'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:24.464873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '314260256dbb4a8e8e2b4c1c5f857dcb', 'request_id': 'req-2e4a1ae9-2767-42f3-8104-f409d369a2b2'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:24.612207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] The token cache is being invalidate because project 314260256dbb4a8e8e2b4c1c5f857dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:24.612542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:24.613449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:24.628276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1039/2078] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:24 2022] DELETE /identity/v3/projects/314260256dbb4a8e8e2b4c1c5f857dcb => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:24.632677 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6763195-8a6d-4dd1-95fe-6ef7ff604940 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:24.645508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6763195-8a6d-4dd1-95fe-6ef7ff604940 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:24.682355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:24.782585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:24.782585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:24.782585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:24.782585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] PATH_INFO: `/v3/users/29fcd54096ab4e688a48a5e857fbfb89` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:24.783059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] RBAC: Authorizing `identity:delete_user(user_id=29fcd54096ab4e688a48a5e857fbfb89)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:24.784952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:25.063431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29fcd54096ab4e688a48a5e857fbfb89'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:25.079932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29fcd54096ab4e688a48a5e857fbfb89'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:25.098541 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29fcd54096ab4e688a48a5e857fbfb89'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:25.154786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29fcd54096ab4e688a48a5e857fbfb89'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:25.155514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a4501a6d-60fe-4199-948c-533a1d40592d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29fcd54096ab4e688a48a5e857fbfb89'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:25.163721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1040/2079] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:24 2022] DELETE /identity/v3/users/29fcd54096ab4e688a48a5e857fbfb89 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:25.614890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:25.616549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:25.647145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a1ae9-2767-42f3-8104-f409d369a2b2 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:25.715708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:25.717222 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:25.717878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:25.718304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] PATH_INFO: `/v3/projects/04d1a61e44984ec4a76c6c099dc4a72f` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:25.720367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d1a61e44984ec4a76c6c099dc4a72f)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:25.722726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:25.738293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:25.877642 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d1a61e44984ec4a76c6c099dc4a72f', 'request_id': 'req-91a2e29e-bc02-41f8-9d69-e707598e186f'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:25.886230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d1a61e44984ec4a76c6c099dc4a72f', 'request_id': 'req-91a2e29e-bc02-41f8-9d69-e707598e186f'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:25.893791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d1a61e44984ec4a76c6c099dc4a72f', 'request_id': 'req-91a2e29e-bc02-41f8-9d69-e707598e186f'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:25.985367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] The token cache is being invalidate because project 04d1a61e44984ec4a76c6c099dc4a72f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:25.988494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:25.988494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91a2e29e-bc02-41f8-9d69-e707598e186f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:25.990632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1040/2080] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:25 2022] DELETE /identity/v3/projects/04d1a61e44984ec4a76c6c099dc4a72f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:25.997881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d5394-bf20-464f-a5d1-729da3703de7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:25.998383 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d5394-bf20-464f-a5d1-729da3703de7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:25.998492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d5394-bf20-464f-a5d1-729da3703de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:26.027335 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f83d5394-bf20-464f-a5d1-729da3703de7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:26.031049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f83d5394-bf20-464f-a5d1-729da3703de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:26.116121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1041/2081] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:45:26.125324 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a503ca-4689-4208-a20b-51c876462ae4 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:26.125621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a503ca-4689-4208-a20b-51c876462ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:26.125899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a503ca-4689-4208-a20b-51c876462ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:26.144083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-65a503ca-4689-4208-a20b-51c876462ae4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:26.145381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-65a503ca-4689-4208-a20b-51c876462ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:26.154113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1041/2082] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:26.158348 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d5394-bf20-464f-a5d1-729da3703de7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:26.170338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d5394-bf20-464f-a5d1-729da3703de7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:26.176081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f19a-784c-4d82-bb6d-88f23de0a791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:26.176658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f19a-784c-4d82-bb6d-88f23de0a791 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:26.176971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f19a-784c-4d82-bb6d-88f23de0a791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:26.178186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f19a-784c-4d82-bb6d-88f23de0a791 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:26.179658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f19a-784c-4d82-bb6d-88f23de0a791 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:45:26.180133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f19a-784c-4d82-bb6d-88f23de0a791 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:26.181818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f19a-784c-4d82-bb6d-88f23de0a791 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:26.203999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1042/2083] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:45:26.274166 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a503ca-4689-4208-a20b-51c876462ae4 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:26.292429 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a503ca-4689-4208-a20b-51c876462ae4 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:26.320268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a63619-5332-409c-81c1-504fe354545f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:26.321266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a63619-5332-409c-81c1-504fe354545f admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:26.321905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a63619-5332-409c-81c1-504fe354545f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:26.322374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a63619-5332-409c-81c1-504fe354545f admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:26.324361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a63619-5332-409c-81c1-504fe354545f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:26.326509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a63619-5332-409c-81c1-504fe354545f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:26.339967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f19a-784c-4d82-bb6d-88f23de0a791 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:26.342881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-70f4f19a-784c-4d82-bb6d-88f23de0a791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:26.367264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f19a-784c-4d82-bb6d-88f23de0a791 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:26.374553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:26.375654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:26.375785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:26.376097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] PATH_INFO: `/v3/users/13f9ef55d1114892b67c812a74daca22` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:26.378545 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f9ef55d1114892b67c812a74daca22)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:26.379792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:26.408092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:26.751057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1042/2084] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:26 2022] POST /identity/v3/projects => generated 373 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:45:26.754623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a63619-5332-409c-81c1-504fe354545f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:26.833352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a63619-5332-409c-81c1-504fe354545f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:26.835283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f9ef55d1114892b67c812a74daca22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:26.839473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab72f91-e8be-462a-bb6e-942bffaec770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:26.840873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab72f91-e8be-462a-bb6e-942bffaec770 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:26.840873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab72f91-e8be-462a-bb6e-942bffaec770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:26.841486 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab72f91-e8be-462a-bb6e-942bffaec770 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:26.842215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab72f91-e8be-462a-bb6e-942bffaec770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:26.844007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f9ef55d1114892b67c812a74daca22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:26.844174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab72f91-e8be-462a-bb6e-942bffaec770 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:26.863910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f9ef55d1114892b67c812a74daca22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:26.932726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1043/2085] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:26 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:45:26.936357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab72f91-e8be-462a-bb6e-942bffaec770 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:26.948522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab72f91-e8be-462a-bb6e-942bffaec770 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:26.956857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f9ef55d1114892b67c812a74daca22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:26.958782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-857cb769-3d2e-40ac-a621-bdc9a0ac26af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f9ef55d1114892b67c812a74daca22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:26.967006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1043/2086] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:26 2022] DELETE /identity/v3/users/13f9ef55d1114892b67c812a74daca22 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:26.972181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c845b-59fc-42fd-8eb2-8b55b42faf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:26.973199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c845b-59fc-42fd-8eb2-8b55b42faf17 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:26.973650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c845b-59fc-42fd-8eb2-8b55b42faf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:26.973931 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b860d-1d7b-4a0c-bcd2-76ac05ec1e5b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:26.974329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b860d-1d7b-4a0c-bcd2-76ac05ec1e5b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:26.974554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c845b-59fc-42fd-8eb2-8b55b42faf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:26.974804 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b860d-1d7b-4a0c-bcd2-76ac05ec1e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:26.975631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c845b-59fc-42fd-8eb2-8b55b42faf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:26.977910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c845b-59fc-42fd-8eb2-8b55b42faf17 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:26.992725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1044/2087] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:45:26.997096 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c845b-59fc-42fd-8eb2-8b55b42faf17 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:26.998800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-3b1b860d-1d7b-4a0c-bcd2-76ac05ec1e5b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:26.999681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8b0c845b-59fc-42fd-8eb2-8b55b42faf17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:27.000275 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3b1b860d-1d7b-4a0c-bcd2-76ac05ec1e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:27.013941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c845b-59fc-42fd-8eb2-8b55b42faf17 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:27.096624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1044/2088] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:27.104560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b860d-1d7b-4a0c-bcd2-76ac05ec1e5b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:27.106809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3b1b860d-1d7b-4a0c-bcd2-76ac05ec1e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:27.110868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94edf4-af84-48fb-add0-2590ce0ac89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:27.112467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94edf4-af84-48fb-add0-2590ce0ac89b admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:27.113516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94edf4-af84-48fb-add0-2590ce0ac89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:27.113976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94edf4-af84-48fb-add0-2590ce0ac89b admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:27.114965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94edf4-af84-48fb-add0-2590ce0ac89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:27.117596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94edf4-af84-48fb-add0-2590ce0ac89b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:27.123625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:27.141797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1045/2089] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:27.145967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94edf4-af84-48fb-add0-2590ce0ac89b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:27.160138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94edf4-af84-48fb-add0-2590ce0ac89b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:27.166583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c16d5-42c3-4b4a-9127-7dcfacdaac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:27.172154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c16d5-42c3-4b4a-9127-7dcfacdaac85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:27.172154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c16d5-42c3-4b4a-9127-7dcfacdaac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:27.172154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c16d5-42c3-4b4a-9127-7dcfacdaac85 admin admin] PATH_INFO: `/v3/projects/ca23768669a74586a221c14ebaa8ee26/users/d352cf5c00ae450c9a7ca4dcc68a87db/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:27.182327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b860d-1d7b-4a0c-bcd2-76ac05ec1e5b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:27.187420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a86b3-42a5-4c25-b804-b5ebe3f99264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:27.188202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a86b3-42a5-4c25-b804-b5ebe3f99264 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:27.188365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a86b3-42a5-4c25-b804-b5ebe3f99264 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:27.188851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a86b3-42a5-4c25-b804-b5ebe3f99264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:27.192147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c16d5-42c3-4b4a-9127-7dcfacdaac85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca23768669a74586a221c14ebaa8ee26, user_id=d352cf5c00ae450c9a7ca4dcc68a87db, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:27.195109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c16d5-42c3-4b4a-9127-7dcfacdaac85 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:27.198763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a86b3-42a5-4c25-b804-b5ebe3f99264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:27.200673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a86b3-42a5-4c25-b804-b5ebe3f99264 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:27.210499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1045/2090] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:27.295210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1046/2091] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:27 2022] PUT /identity/v3/projects/ca23768669a74586a221c14ebaa8ee26/users/d352cf5c00ae450c9a7ca4dcc68a87db/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:27.301588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be7af7-9808-43b3-9a86-d6decf07d9cd None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:27.302155 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be7af7-9808-43b3-9a86-d6decf07d9cd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:27.302539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be7af7-9808-43b3-9a86-d6decf07d9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:27.327119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-40be7af7-9808-43b3-9a86-d6decf07d9cd None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:27.328367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-40be7af7-9808-43b3-9a86-d6decf07d9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:27.403553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1046/2092] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:45:27.414232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c16d5-42c3-4b4a-9127-7dcfacdaac85 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:27.489650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c16d5-42c3-4b4a-9127-7dcfacdaac85 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:27.495792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecfb00-2251-48f3-81f9-b87690a43d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:27.496637 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecfb00-2251-48f3-81f9-b87690a43d2c service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:27.496931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecfb00-2251-48f3-81f9-b87690a43d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:27.497352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecfb00-2251-48f3-81f9-b87690a43d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:27.509940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecfb00-2251-48f3-81f9-b87690a43d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:27.511590 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecfb00-2251-48f3-81f9-b87690a43d2c service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:27.541634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1047/2093] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:45:27.688636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:27.735184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be7af7-9808-43b3-9a86-d6decf07d9cd None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:27.737200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-40be7af7-9808-43b3-9a86-d6decf07d9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:27.753642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be7af7-9808-43b3-9a86-d6decf07d9cd None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:27.759606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:27.760747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:27.762181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:27.762483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] PATH_INFO: `/v3/projects/3e42823305674a07acd1281462b6d68f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:27.766588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e42823305674a07acd1281462b6d68f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:27.779168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:28.000751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e42823305674a07acd1281462b6d68f', 'request_id': 'req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:28.072234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e42823305674a07acd1281462b6d68f', 'request_id': 'req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:28.082496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e42823305674a07acd1281462b6d68f', 'request_id': 'req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:28.097122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] The token cache is being invalidate because project 3e42823305674a07acd1281462b6d68f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:28.097582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:28.098448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:28.103584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1047/2094] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:27 2022] DELETE /identity/v3/projects/3e42823305674a07acd1281462b6d68f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:28.104723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecfb00-2251-48f3-81f9-b87690a43d2c service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:28.120252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecfb00-2251-48f3-81f9-b87690a43d2c service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:28.209953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:28.215876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:28.216535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:28.216931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] PATH_INFO: `/v3/users/cb553cd529884668a0dde97744f8fef1` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:28.219602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb553cd529884668a0dde97744f8fef1)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:28.220624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:28.469437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb553cd529884668a0dde97744f8fef1'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:28.470251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb553cd529884668a0dde97744f8fef1'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:28.485999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb553cd529884668a0dde97744f8fef1'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:28.488579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:28.500210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb553cd529884668a0dde97744f8fef1'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:28.574302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb553cd529884668a0dde97744f8fef1'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:28.584480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1048/2095] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:28 2022] DELETE /identity/v3/users/cb553cd529884668a0dde97744f8fef1 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:29.136136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:29.137726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:29.154179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e70c27-61db-4ccb-a00b-f6d9e9621e2c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:29.216508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:29.217478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:29.217760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:29.218045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] PATH_INFO: `/v3/projects/fd1e2f000c534c1b8f2036a3b9bf107d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:29.219367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd1e2f000c534c1b8f2036a3b9bf107d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:29.221324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:29.365902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd1e2f000c534c1b8f2036a3b9bf107d', 'request_id': 'req-69574038-a214-40ae-b1aa-f2b53326d365'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:29.462323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd1e2f000c534c1b8f2036a3b9bf107d', 'request_id': 'req-69574038-a214-40ae-b1aa-f2b53326d365'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:29.469091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd1e2f000c534c1b8f2036a3b9bf107d', 'request_id': 'req-69574038-a214-40ae-b1aa-f2b53326d365'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:29.471163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:29.481034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] The token cache is being invalidate because project fd1e2f000c534c1b8f2036a3b9bf107d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:29.481541 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:29.482258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:29.483991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1048/2096] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:29 2022] DELETE /identity/v3/projects/fd1e2f000c534c1b8f2036a3b9bf107d => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:29.487983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:29.489769 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:29.503026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a3fa5-04fa-43fd-b41a-408b002eb1c6 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:29.570305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:29.570715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:29.571000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:29.571273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] PATH_INFO: `/v3/users/bd18fc9e7ab2441180c17f1b87ba2068` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:29.572529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd18fc9e7ab2441180c17f1b87ba2068)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:29.575077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:29.842964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd18fc9e7ab2441180c17f1b87ba2068'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:29.843762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd18fc9e7ab2441180c17f1b87ba2068'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:29.873102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd18fc9e7ab2441180c17f1b87ba2068'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:29.888908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd18fc9e7ab2441180c17f1b87ba2068'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:30.023179 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2370d8a7-fbce-4d95-8c8b-8858dc4f1460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd18fc9e7ab2441180c17f1b87ba2068'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:30.035139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1049/2097] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:29 2022] DELETE /identity/v3/users/bd18fc9e7ab2441180c17f1b87ba2068 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:31.324456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:31.326427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:31.356109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69574038-a214-40ae-b1aa-f2b53326d365 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:31.469649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:31.470654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:31.471002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:31.471342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] PATH_INFO: `/v3/projects/ad8e1d9f98e349ac8004d00d4dbb23c3` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:31.472694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8e1d9f98e349ac8004d00d4dbb23c3)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:31.474741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:31.601605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8e1d9f98e349ac8004d00d4dbb23c3', 'request_id': 'req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:31.693712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8e1d9f98e349ac8004d00d4dbb23c3', 'request_id': 'req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:31.701049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8e1d9f98e349ac8004d00d4dbb23c3', 'request_id': 'req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:31.713734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] The token cache is being invalidate because project ad8e1d9f98e349ac8004d00d4dbb23c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:31.714197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:31.715379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b906a9bd-f5fa-4a26-8f40-92ce28f420b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:31.716955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1049/2098] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:31 2022] DELETE /identity/v3/projects/ad8e1d9f98e349ac8004d00d4dbb23c3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:31.722252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99001d7f-37cf-477d-892c-97ef9149d216 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:31.722737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99001d7f-37cf-477d-892c-97ef9149d216 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:31.722847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99001d7f-37cf-477d-892c-97ef9149d216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:31.748798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-99001d7f-37cf-477d-892c-97ef9149d216 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:31.748798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-99001d7f-37cf-477d-892c-97ef9149d216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:31.813684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1050/2099] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:31.822548 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d087a7-0bd2-4e5b-810c-f8e68e18d2b7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:31.822841 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d087a7-0bd2-4e5b-810c-f8e68e18d2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:31.823119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d087a7-0bd2-4e5b-810c-f8e68e18d2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:31.852600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-42d087a7-0bd2-4e5b-810c-f8e68e18d2b7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:31.853902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-42d087a7-0bd2-4e5b-810c-f8e68e18d2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:31.867528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1050/2100] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:31.867763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99001d7f-37cf-477d-892c-97ef9149d216 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:31.873944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:31.878253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99001d7f-37cf-477d-892c-97ef9149d216 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:31.883364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9539ad-e1a0-47d8-8934-1e82a061a8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:31.888280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9539ad-e1a0-47d8-8934-1e82a061a8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:31.888280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9539ad-e1a0-47d8-8934-1e82a061a8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:31.888280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9539ad-e1a0-47d8-8934-1e82a061a8d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:31.888280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9539ad-e1a0-47d8-8934-1e82a061a8d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:45:31.888280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9539ad-e1a0-47d8-8934-1e82a061a8d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:31.888280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9539ad-e1a0-47d8-8934-1e82a061a8d6 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:31.902454 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1051/2101] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:45:31.906861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d087a7-0bd2-4e5b-810c-f8e68e18d2b7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:31.917475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d087a7-0bd2-4e5b-810c-f8e68e18d2b7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:31.923267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b3385-86b8-48be-baba-ce0cd97f9e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:31.923946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b3385-86b8-48be-baba-ce0cd97f9e42 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:31.924556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b3385-86b8-48be-baba-ce0cd97f9e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:31.924556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b3385-86b8-48be-baba-ce0cd97f9e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:31.926563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b3385-86b8-48be-baba-ce0cd97f9e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:31.929792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b3385-86b8-48be-baba-ce0cd97f9e42 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:32.050900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1051/2102] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:31 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:32.055191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9539ad-e1a0-47d8-8934-1e82a061a8d6 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:32.068092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9539ad-e1a0-47d8-8934-1e82a061a8d6 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:32.153070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994ff3d-b69e-4d4a-9a63-1dc533a48129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:32.153994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994ff3d-b69e-4d4a-9a63-1dc533a48129 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:32.154423 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994ff3d-b69e-4d4a-9a63-1dc533a48129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:32.155029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994ff3d-b69e-4d4a-9a63-1dc533a48129 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:32.156950 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994ff3d-b69e-4d4a-9a63-1dc533a48129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:32.160322 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994ff3d-b69e-4d4a-9a63-1dc533a48129 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:32.276099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1052/2103] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:32 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:45:32.280028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b3385-86b8-48be-baba-ce0cd97f9e42 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:32.291633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b3385-86b8-48be-baba-ce0cd97f9e42 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:32.299051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c71424-92e5-41a5-bed9-b9d3a67cd0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:32.299860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c71424-92e5-41a5-bed9-b9d3a67cd0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:32.300196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c71424-92e5-41a5-bed9-b9d3a67cd0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:32.320658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c71424-92e5-41a5-bed9-b9d3a67cd0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:32.320658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c71424-92e5-41a5-bed9-b9d3a67cd0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:32.320658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c71424-92e5-41a5-bed9-b9d3a67cd0c1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:32.320658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1052/2104] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:32.325001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994ff3d-b69e-4d4a-9a63-1dc533a48129 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:32.339028 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994ff3d-b69e-4d4a-9a63-1dc533a48129 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:32.343875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc011391-1723-43c6-8c68-19755879ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:32.344648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc011391-1723-43c6-8c68-19755879ca23 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:32.345037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc011391-1723-43c6-8c68-19755879ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:32.350062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc011391-1723-43c6-8c68-19755879ca23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:32.350062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc011391-1723-43c6-8c68-19755879ca23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:32.350062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc011391-1723-43c6-8c68-19755879ca23 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:32.359597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1053/2105] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:45:32.363488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c71424-92e5-41a5-bed9-b9d3a67cd0c1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:32.375509 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c71424-92e5-41a5-bed9-b9d3a67cd0c1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:32.380976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30670dbe-832d-4699-bb45-12d832a83d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:32.381996 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30670dbe-832d-4699-bb45-12d832a83d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:32.382470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30670dbe-832d-4699-bb45-12d832a83d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:32.382897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30670dbe-832d-4699-bb45-12d832a83d88 admin admin] PATH_INFO: `/v3/projects/db52f8595c2d43c89cc0ea5ae5dbcf9e/users/a1a44f7880dd41d58e78d7ca5de019bd/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:32.402390 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30670dbe-832d-4699-bb45-12d832a83d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db52f8595c2d43c89cc0ea5ae5dbcf9e, user_id=a1a44f7880dd41d58e78d7ca5de019bd, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:32.405465 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30670dbe-832d-4699-bb45-12d832a83d88 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:32.491142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1053/2106] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:32 2022] PUT /identity/v3/projects/db52f8595c2d43c89cc0ea5ae5dbcf9e/users/a1a44f7880dd41d58e78d7ca5de019bd/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:32.494670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2c871-e847-4f67-9252-8101567ac601 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:32.495071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2c871-e847-4f67-9252-8101567ac601 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:32.496419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2c871-e847-4f67-9252-8101567ac601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:32.535162 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-57b2c871-e847-4f67-9252-8101567ac601 None None] MFA Rules not processed for user `a1a44f7880dd41d58e78d7ca5de019bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:32.536411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-57b2c871-e847-4f67-9252-8101567ac601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:32.652655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1054/2107] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:32.665335 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30670dbe-832d-4699-bb45-12d832a83d88 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:32.667416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-30670dbe-832d-4699-bb45-12d832a83d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:32.707478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:32.748500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30670dbe-832d-4699-bb45-12d832a83d88 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:32.753230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4550ac-b268-4ac9-b1f4-0ad57154e09b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:32.754216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4550ac-b268-4ac9-b1f4-0ad57154e09b service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:32.754729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4550ac-b268-4ac9-b1f4-0ad57154e09b service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:32.755148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4550ac-b268-4ac9-b1f4-0ad57154e09b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:32.767346 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4550ac-b268-4ac9-b1f4-0ad57154e09b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:32.770004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4550ac-b268-4ac9-b1f4-0ad57154e09b service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:32.781287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1054/2108] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:32.824091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2c871-e847-4f67-9252-8101567ac601 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:32.928950 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2c871-e847-4f67-9252-8101567ac601 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:32.935657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:32.936913 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:32.937231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:32.937512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] PATH_INFO: `/v3/users/a1a44f7880dd41d58e78d7ca5de019bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:32.940364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a44f7880dd41d58e78d7ca5de019bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:32.942584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:32.944694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:33.235670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a44f7880dd41d58e78d7ca5de019bd'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:33.236818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a44f7880dd41d58e78d7ca5de019bd'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:33.253770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a44f7880dd41d58e78d7ca5de019bd'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:33.263763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a44f7880dd41d58e78d7ca5de019bd'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:33.419817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a44f7880dd41d58e78d7ca5de019bd'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:33.435398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1055/2109] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:32 2022] DELETE /identity/v3/users/a1a44f7880dd41d58e78d7ca5de019bd => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:33.436375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0fb2b-8e68-4885-a7ee-f568ce3c9d52 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:33.436858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0fb2b-8e68-4885-a7ee-f568ce3c9d52 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:33.436961 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0fb2b-8e68-4885-a7ee-f568ce3c9d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:33.455776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-abe0fb2b-8e68-4885-a7ee-f568ce3c9d52 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:33.457003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-abe0fb2b-8e68-4885-a7ee-f568ce3c9d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:33.544211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1055/2110] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:45:33.554279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:33.556068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:33.655006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d62dd0-8bda-483b-b0ee-f291a8108ccd admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:33.659144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb05c5-66ca-4c95-90b0-ea3d48bf9eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:33.660385 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb05c5-66ca-4c95-90b0-ea3d48bf9eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:33.660621 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb05c5-66ca-4c95-90b0-ea3d48bf9eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:33.660893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb05c5-66ca-4c95-90b0-ea3d48bf9eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:33.673510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb05c5-66ca-4c95-90b0-ea3d48bf9eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:33.675451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb05c5-66ca-4c95-90b0-ea3d48bf9eb0 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:33.689414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1056/2111] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:33.746146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb952a59-ec0a-424c-966d-23ab90c9355a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:33.746608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb952a59-ec0a-424c-966d-23ab90c9355a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:33.747650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb952a59-ec0a-424c-966d-23ab90c9355a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:33.789212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cb952a59-ec0a-424c-966d-23ab90c9355a None None] MFA Rules not processed for user `d352cf5c00ae450c9a7ca4dcc68a87db`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:33.790306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cb952a59-ec0a-424c-966d-23ab90c9355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:33.941443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1056/2112] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:33.945735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb05c5-66ca-4c95-90b0-ea3d48bf9eb0 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:33.947697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-80cb05c5-66ca-4c95-90b0-ea3d48bf9eb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:33.962762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb05c5-66ca-4c95-90b0-ea3d48bf9eb0 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:33.968910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41924b7-45d8-4f7f-9dc9-5ca9340f9233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:33.969373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41924b7-45d8-4f7f-9dc9-5ca9340f9233 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:33.969845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41924b7-45d8-4f7f-9dc9-5ca9340f9233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:33.970279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41924b7-45d8-4f7f-9dc9-5ca9340f9233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:33.971000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41924b7-45d8-4f7f-9dc9-5ca9340f9233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:33.972927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41924b7-45d8-4f7f-9dc9-5ca9340f9233 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:34.075214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1057/2113] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:33 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:34.079787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb952a59-ec0a-424c-966d-23ab90c9355a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:34.093319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb952a59-ec0a-424c-966d-23ab90c9355a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:34.178853 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e43d45-87aa-430f-893a-55ab2a1de0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:34.179551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e43d45-87aa-430f-893a-55ab2a1de0aa admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:34.179845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e43d45-87aa-430f-893a-55ab2a1de0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:34.180861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e43d45-87aa-430f-893a-55ab2a1de0aa admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:34.181947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e43d45-87aa-430f-893a-55ab2a1de0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:34.184892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e43d45-87aa-430f-893a-55ab2a1de0aa admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:34.334440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1057/2114] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:34 2022] POST /identity/v3/users => generated 292 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:34.342313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41924b7-45d8-4f7f-9dc9-5ca9340f9233 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:34.365725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41924b7-45d8-4f7f-9dc9-5ca9340f9233 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:34.378171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53963817-e0dd-48df-8029-d851f560fa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:34.378863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53963817-e0dd-48df-8029-d851f560fa4d admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:34.379307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53963817-e0dd-48df-8029-d851f560fa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:34.379590 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53963817-e0dd-48df-8029-d851f560fa4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:34.380380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53963817-e0dd-48df-8029-d851f560fa4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:34.382029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53963817-e0dd-48df-8029-d851f560fa4d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:34.396939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e43d45-87aa-430f-893a-55ab2a1de0aa admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:34.400075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e6e43d45-87aa-430f-893a-55ab2a1de0aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:34.400736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1058/2115] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:34.407887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53963817-e0dd-48df-8029-d851f560fa4d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:34.420829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53963817-e0dd-48df-8029-d851f560fa4d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:34.433606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4bd66-45c7-4e84-9448-57be041005fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:34.434707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4bd66-45c7-4e84-9448-57be041005fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:34.435192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4bd66-45c7-4e84-9448-57be041005fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:34.435485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4bd66-45c7-4e84-9448-57be041005fd admin admin] PATH_INFO: `/v3/projects/a0070026dba04010b93def3ca14598ea/users/ad07006492bb407da09a6b5387237f55/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:34.440149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e43d45-87aa-430f-893a-55ab2a1de0aa admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:34.446227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:34.447606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:34.449310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:34.449754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] PATH_INFO: `/v3/projects/db52f8595c2d43c89cc0ea5ae5dbcf9e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:34.451340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=db52f8595c2d43c89cc0ea5ae5dbcf9e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:34.454157 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:34.461532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4bd66-45c7-4e84-9448-57be041005fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0070026dba04010b93def3ca14598ea, user_id=ad07006492bb407da09a6b5387237f55, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:34.465642 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4bd66-45c7-4e84-9448-57be041005fd admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:34.534901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1059/2116] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:34 2022] PUT /identity/v3/projects/a0070026dba04010b93def3ca14598ea/users/ad07006492bb407da09a6b5387237f55/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:34.538762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4bd66-45c7-4e84-9448-57be041005fd admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:34.579060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db52f8595c2d43c89cc0ea5ae5dbcf9e', 'request_id': 'req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:34.631798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4bd66-45c7-4e84-9448-57be041005fd admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:34.684507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db52f8595c2d43c89cc0ea5ae5dbcf9e', 'request_id': 'req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:34.693447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db52f8595c2d43c89cc0ea5ae5dbcf9e', 'request_id': 'req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:34.707184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] The token cache is being invalidate because project db52f8595c2d43c89cc0ea5ae5dbcf9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:34.707749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:34.708617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4ea36038-4a46-42ac-9eb9-153d6b5ba4dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:34.710805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1058/2117] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:34 2022] DELETE /identity/v3/projects/db52f8595c2d43c89cc0ea5ae5dbcf9e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:34.715819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8908ba4-dca9-44cc-b86e-9b3a978b6453 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:34.717053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8908ba4-dca9-44cc-b86e-9b3a978b6453 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:34.721149 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8908ba4-dca9-44cc-b86e-9b3a978b6453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:34.727140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09704799-e19d-4c62-864c-dae5f03ecb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:34.727827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09704799-e19d-4c62-864c-dae5f03ecb9b admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:34.728110 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09704799-e19d-4c62-864c-dae5f03ecb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:34.728382 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09704799-e19d-4c62-864c-dae5f03ecb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:34.729124 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09704799-e19d-4c62-864c-dae5f03ecb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:34.732308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09704799-e19d-4c62-864c-dae5f03ecb9b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:34.744405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d8908ba4-dca9-44cc-b86e-9b3a978b6453 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:34.746901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d8908ba4-dca9-44cc-b86e-9b3a978b6453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:34.755929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1060/2118] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:34.760064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09704799-e19d-4c62-864c-dae5f03ecb9b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:34.761528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1059/2119] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:34.771146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-09704799-e19d-4c62-864c-dae5f03ecb9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:34.771556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dd65d-0044-49e1-a938-7a77b912e279 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:34.772052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dd65d-0044-49e1-a938-7a77b912e279 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:34.772490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dd65d-0044-49e1-a938-7a77b912e279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:34.788232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09704799-e19d-4c62-864c-dae5f03ecb9b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:34.794092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b633b5-4fca-438a-8bde-779f8d5212d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:34.795184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b633b5-4fca-438a-8bde-779f8d5212d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:34.795588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b633b5-4fca-438a-8bde-779f8d5212d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:34.796494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b633b5-4fca-438a-8bde-779f8d5212d8 admin admin] PATH_INFO: `/v3/projects/a0070026dba04010b93def3ca14598ea/users/ad07006492bb407da09a6b5387237f55/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:34.799879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d78dd65d-0044-49e1-a938-7a77b912e279 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:34.800224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b633b5-4fca-438a-8bde-779f8d5212d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0070026dba04010b93def3ca14598ea, user_id=ad07006492bb407da09a6b5387237f55, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:34.801953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d78dd65d-0044-49e1-a938-7a77b912e279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:34.809102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b633b5-4fca-438a-8bde-779f8d5212d8 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:34.815188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1060/2120] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:34.819048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dd65d-0044-49e1-a938-7a77b912e279 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:34.828721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1061/2121] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:34 2022] PUT /identity/v3/projects/a0070026dba04010b93def3ca14598ea/users/ad07006492bb407da09a6b5387237f55/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:45:34.834529 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dd65d-0044-49e1-a938-7a77b912e279 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:34.912421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af3c41-c475-447e-a92d-47e01ee20b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:34.912421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af3c41-c475-447e-a92d-47e01ee20b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:34.912421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af3c41-c475-447e-a92d-47e01ee20b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:34.912421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af3c41-c475-447e-a92d-47e01ee20b37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:34.913139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af3c41-c475-447e-a92d-47e01ee20b37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:45:34.913139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af3c41-c475-447e-a92d-47e01ee20b37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:34.913139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af3c41-c475-447e-a92d-47e01ee20b37 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:34.931942 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1061/2122] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:45:34.936489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b633b5-4fca-438a-8bde-779f8d5212d8 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:34.949465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b633b5-4fca-438a-8bde-779f8d5212d8 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:34.962469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29f5a6-81a4-420e-b625-fcbd7ab18824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:34.962469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29f5a6-81a4-420e-b625-fcbd7ab18824 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:34.962469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29f5a6-81a4-420e-b625-fcbd7ab18824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:34.962469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29f5a6-81a4-420e-b625-fcbd7ab18824 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:34.962835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29f5a6-81a4-420e-b625-fcbd7ab18824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:34.962835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29f5a6-81a4-420e-b625-fcbd7ab18824 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:35.107042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1062/2123] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:34 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:35.110162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af3c41-c475-447e-a92d-47e01ee20b37 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:35.124026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af3c41-c475-447e-a92d-47e01ee20b37 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:35.200460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9f44b-7c9e-446a-be2c-9c3fec703ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:35.201146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9f44b-7c9e-446a-be2c-9c3fec703ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:35.201458 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9f44b-7c9e-446a-be2c-9c3fec703ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:35.201736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9f44b-7c9e-446a-be2c-9c3fec703ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:35.202437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9f44b-7c9e-446a-be2c-9c3fec703ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:35.204700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9f44b-7c9e-446a-be2c-9c3fec703ae2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:35.303135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1062/2124] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:35 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:35.303810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29f5a6-81a4-420e-b625-fcbd7ab18824 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:35.316682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29f5a6-81a4-420e-b625-fcbd7ab18824 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:35.322052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cc7b4-217a-43f6-817a-4aa3fe85b9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:35.323164 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cc7b4-217a-43f6-817a-4aa3fe85b9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:35.324150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cc7b4-217a-43f6-817a-4aa3fe85b9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:35.324943 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cc7b4-217a-43f6-817a-4aa3fe85b9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:35.325908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39cc7b4-217a-43f6-817a-4aa3fe85b9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:35.327775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39cc7b4-217a-43f6-817a-4aa3fe85b9b2 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:35.343746 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1063/2125] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:45:35.347794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9f44b-7c9e-446a-be2c-9c3fec703ae2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:35.359535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9f44b-7c9e-446a-be2c-9c3fec703ae2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:35.364115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefd6a6-52cd-4457-bf97-1f715640c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:35.364776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefd6a6-52cd-4457-bf97-1f715640c4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:35.365056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefd6a6-52cd-4457-bf97-1f715640c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:35.365394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefd6a6-52cd-4457-bf97-1f715640c4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:35.366135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefd6a6-52cd-4457-bf97-1f715640c4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:35.367737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefd6a6-52cd-4457-bf97-1f715640c4e7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:35.378313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1063/2126] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:35.382638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cc7b4-217a-43f6-817a-4aa3fe85b9b2 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:35.394714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cc7b4-217a-43f6-817a-4aa3fe85b9b2 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:35.399881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89416e73-1cf3-44f0-8e67-d11d3aaf5b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:35.400772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89416e73-1cf3-44f0-8e67-d11d3aaf5b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:35.401057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89416e73-1cf3-44f0-8e67-d11d3aaf5b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:35.401366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89416e73-1cf3-44f0-8e67-d11d3aaf5b8e admin admin] PATH_INFO: `/v3/projects/1ef60975c92640fe9b7f96d5f6236b8d/users/a5147f8dd28a4ababbc4886861f2b6f6/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:35.418533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89416e73-1cf3-44f0-8e67-d11d3aaf5b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef60975c92640fe9b7f96d5f6236b8d, user_id=a5147f8dd28a4ababbc4886861f2b6f6, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:35.421461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89416e73-1cf3-44f0-8e67-d11d3aaf5b8e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:35.498037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:35.506792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1064/2127] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:35 2022] PUT /identity/v3/projects/1ef60975c92640fe9b7f96d5f6236b8d/users/a5147f8dd28a4ababbc4886861f2b6f6/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:35.580159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16e845-4fb6-4dff-b2bc-0c91473fb5d4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:35.580790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16e845-4fb6-4dff-b2bc-0c91473fb5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:35.580790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16e845-4fb6-4dff-b2bc-0c91473fb5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:35.610827 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0b16e845-4fb6-4dff-b2bc-0c91473fb5d4 None None] MFA Rules not processed for user `a5147f8dd28a4ababbc4886861f2b6f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:35.612074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0b16e845-4fb6-4dff-b2bc-0c91473fb5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:35.678454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1064/2128] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:35.690675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89416e73-1cf3-44f0-8e67-d11d3aaf5b8e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:35.692412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-89416e73-1cf3-44f0-8e67-d11d3aaf5b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:35.756053 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89416e73-1cf3-44f0-8e67-d11d3aaf5b8e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:35.760847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993b233-0265-4f67-b0f1-b07594cb3029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:35.761606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993b233-0265-4f67-b0f1-b07594cb3029 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:35.766240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993b233-0265-4f67-b0f1-b07594cb3029 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:35.766240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993b233-0265-4f67-b0f1-b07594cb3029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:35.771304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993b233-0265-4f67-b0f1-b07594cb3029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:35.772837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993b233-0265-4f67-b0f1-b07594cb3029 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:35.785271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1065/2129] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:45:35.863097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:36.645797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16e845-4fb6-4dff-b2bc-0c91473fb5d4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:36.648299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0b16e845-4fb6-4dff-b2bc-0c91473fb5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:36.736352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16e845-4fb6-4dff-b2bc-0c91473fb5d4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:36.743868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:36.744590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:36.744880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:36.745175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] PATH_INFO: `/v3/users/50c31cd004fb48e6b49901bb3885ff5c` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:36.747205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50c31cd004fb48e6b49901bb3885ff5c)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:36.749827 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:37.041048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50c31cd004fb48e6b49901bb3885ff5c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:37.050180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50c31cd004fb48e6b49901bb3885ff5c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:37.064547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50c31cd004fb48e6b49901bb3885ff5c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:37.099867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:37.170980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50c31cd004fb48e6b49901bb3885ff5c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:37.171885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50c31cd004fb48e6b49901bb3885ff5c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:37.185414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1065/2130] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:36 2022] DELETE /identity/v3/users/50c31cd004fb48e6b49901bb3885ff5c => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:37.186160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1f557-d92e-4d71-bb33-e99224d29dc7 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:37.186640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1f557-d92e-4d71-bb33-e99224d29dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:37.186841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1f557-d92e-4d71-bb33-e99224d29dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:37.227631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c7c1f557-d92e-4d71-bb33-e99224d29dc7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:37.228864 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c7c1f557-d92e-4d71-bb33-e99224d29dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:37.367281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1066/2131] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:37.382093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:37.387653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:37.527535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38bc1b-d0ef-47bc-991d-684297153ab8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:37.541311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b92f9c-485f-41a5-b2c4-7b2c6ae9714e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:37.541311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b92f9c-485f-41a5-b2c4-7b2c6ae9714e service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:37.541311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b92f9c-485f-41a5-b2c4-7b2c6ae9714e service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:37.541311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b92f9c-485f-41a5-b2c4-7b2c6ae9714e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:37.553214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b92f9c-485f-41a5-b2c4-7b2c6ae9714e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:37.556328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b92f9c-485f-41a5-b2c4-7b2c6ae9714e service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:37.576736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1066/2132] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:38.382324 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1f557-d92e-4d71-bb33-e99224d29dc7 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:38.384630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c7c1f557-d92e-4d71-bb33-e99224d29dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:38.400051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1f557-d92e-4d71-bb33-e99224d29dc7 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:38.415952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:38.419022 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:38.419696 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:38.420261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] PATH_INFO: `/v3/projects/00cd34c5788d4ac1a4bcff19d26f8d10` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:38.423653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00cd34c5788d4ac1a4bcff19d26f8d10)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:38.427701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:38.575046 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00cd34c5788d4ac1a4bcff19d26f8d10', 'request_id': 'req-6c96557a-88ad-4cd3-ae87-d2a3e4619866'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:38.592662 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00cd34c5788d4ac1a4bcff19d26f8d10', 'request_id': 'req-6c96557a-88ad-4cd3-ae87-d2a3e4619866'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:38.602956 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00cd34c5788d4ac1a4bcff19d26f8d10', 'request_id': 'req-6c96557a-88ad-4cd3-ae87-d2a3e4619866'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:38.735558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] The token cache is being invalidate because project 00cd34c5788d4ac1a4bcff19d26f8d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:38.736080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:38.737662 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6c96557a-88ad-4cd3-ae87-d2a3e4619866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:38.741391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1067/2133] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:38 2022] DELETE /identity/v3/projects/00cd34c5788d4ac1a4bcff19d26f8d10 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:38.746435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ce476-0f5a-403b-aa3c-1e39e76cfd4b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:38.747564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ce476-0f5a-403b-aa3c-1e39e76cfd4b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:38.748062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ce476-0f5a-403b-aa3c-1e39e76cfd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:38.821015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d01ce476-0f5a-403b-aa3c-1e39e76cfd4b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:38.824095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d01ce476-0f5a-403b-aa3c-1e39e76cfd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:39.022133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1067/2134] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:39.024151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ce61c-26b2-4673-8c2a-1747725a7ed9 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:39.024477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ce61c-26b2-4673-8c2a-1747725a7ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:39.024759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ce61c-26b2-4673-8c2a-1747725a7ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:39.059312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-cc9ce61c-26b2-4673-8c2a-1747725a7ed9 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:39.062387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cc9ce61c-26b2-4673-8c2a-1747725a7ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:39.077863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ce476-0f5a-403b-aa3c-1e39e76cfd4b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:39.084107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1068/2135] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:39.093387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ce476-0f5a-403b-aa3c-1e39e76cfd4b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:39.106764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc25ed-aec4-4b2f-81fc-d1166273ee44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:39.108423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc25ed-aec4-4b2f-81fc-d1166273ee44 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:39.109424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc25ed-aec4-4b2f-81fc-d1166273ee44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:39.110725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc25ed-aec4-4b2f-81fc-d1166273ee44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:39.112048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc25ed-aec4-4b2f-81fc-d1166273ee44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:45:39.112533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc25ed-aec4-4b2f-81fc-d1166273ee44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:39.119218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc25ed-aec4-4b2f-81fc-d1166273ee44 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:39.147551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1068/2136] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:45:39.153405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ce61c-26b2-4673-8c2a-1747725a7ed9 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:39.163797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ce61c-26b2-4673-8c2a-1747725a7ed9 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:39.171295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132792a-81db-4b3c-857b-13c0a41d17fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:39.171295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132792a-81db-4b3c-857b-13c0a41d17fb admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:39.171295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132792a-81db-4b3c-857b-13c0a41d17fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:39.171295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132792a-81db-4b3c-857b-13c0a41d17fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:39.171694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132792a-81db-4b3c-857b-13c0a41d17fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:39.173002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132792a-81db-4b3c-857b-13c0a41d17fb admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:39.314204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1069/2137] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:39 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:39.318517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc25ed-aec4-4b2f-81fc-d1166273ee44 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:39.329630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc25ed-aec4-4b2f-81fc-d1166273ee44 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:39.405787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3ac74-6258-4853-ac44-e60926cab29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:39.406815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3ac74-6258-4853-ac44-e60926cab29d admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:39.407169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3ac74-6258-4853-ac44-e60926cab29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:39.407546 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3ac74-6258-4853-ac44-e60926cab29d admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:39.408334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3ac74-6258-4853-ac44-e60926cab29d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:39.410403 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3ac74-6258-4853-ac44-e60926cab29d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:39.529275 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1069/2138] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:39 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:45:39.533480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132792a-81db-4b3c-857b-13c0a41d17fb admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:39.551080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132792a-81db-4b3c-857b-13c0a41d17fb admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:39.564047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62942cb-d3e8-4c0f-960c-4969aa5cfb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:39.565601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62942cb-d3e8-4c0f-960c-4969aa5cfb97 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:39.566027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62942cb-d3e8-4c0f-960c-4969aa5cfb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:39.566378 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62942cb-d3e8-4c0f-960c-4969aa5cfb97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:39.567616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62942cb-d3e8-4c0f-960c-4969aa5cfb97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:39.571542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62942cb-d3e8-4c0f-960c-4969aa5cfb97 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:39.609547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1070/2139] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:39.610269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3ac74-6258-4853-ac44-e60926cab29d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:39.623628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3ac74-6258-4853-ac44-e60926cab29d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:39.647326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cee0d7-138e-4e3f-a0d3-3a04eaa7fc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:39.648517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cee0d7-138e-4e3f-a0d3-3a04eaa7fc73 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:39.648808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cee0d7-138e-4e3f-a0d3-3a04eaa7fc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:39.650087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cee0d7-138e-4e3f-a0d3-3a04eaa7fc73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:39.650889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cee0d7-138e-4e3f-a0d3-3a04eaa7fc73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:39.654926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cee0d7-138e-4e3f-a0d3-3a04eaa7fc73 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:39.689830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1070/2140] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:45:39.694410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62942cb-d3e8-4c0f-960c-4969aa5cfb97 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:39.710430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62942cb-d3e8-4c0f-960c-4969aa5cfb97 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:39.722094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d96e4-ca66-4b15-9d57-66b2696cda1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:39.723211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d96e4-ca66-4b15-9d57-66b2696cda1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:39.723871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d96e4-ca66-4b15-9d57-66b2696cda1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:39.724340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d96e4-ca66-4b15-9d57-66b2696cda1f admin admin] PATH_INFO: `/v3/projects/e32f7f4be6e3422db1bd5c39b3fac909/users/1708ab5cf6b24fad9171f343d540698f/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:39.731509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:39.767730 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d96e4-ca66-4b15-9d57-66b2696cda1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e32f7f4be6e3422db1bd5c39b3fac909, user_id=1708ab5cf6b24fad9171f343d540698f, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:39.775190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d96e4-ca66-4b15-9d57-66b2696cda1f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:39.943160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1071/2141] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:39 2022] PUT /identity/v3/projects/e32f7f4be6e3422db1bd5c39b3fac909/users/1708ab5cf6b24fad9171f343d540698f/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:39.955611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f57b7a-787e-4752-92d6-653cabc748e9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:39.956055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f57b7a-787e-4752-92d6-653cabc748e9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:39.956055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f57b7a-787e-4752-92d6-653cabc748e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:40.015301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-64f57b7a-787e-4752-92d6-653cabc748e9 None None] MFA Rules not processed for user `1708ab5cf6b24fad9171f343d540698f`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:40.031413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-64f57b7a-787e-4752-92d6-653cabc748e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:40.165141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1071/2142] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:40.837796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d96e4-ca66-4b15-9d57-66b2696cda1f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:40.840695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-fa9d96e4-ca66-4b15-9d57-66b2696cda1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:40.942401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d96e4-ca66-4b15-9d57-66b2696cda1f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:40.981039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd86d5-e496-4769-963f-a8d0bc4f3dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:40.984262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd86d5-e496-4769-963f-a8d0bc4f3dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:40.984262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd86d5-e496-4769-963f-a8d0bc4f3dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:40.984262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd86d5-e496-4769-963f-a8d0bc4f3dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:41.009893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd86d5-e496-4769-963f-a8d0bc4f3dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:41.012769 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd86d5-e496-4769-963f-a8d0bc4f3dc7 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:41.042303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1072/2143] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:41.365837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:42.128535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a7e41-3676-4e41-981d-1b823b8c94c4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:42.129046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a7e41-3676-4e41-981d-1b823b8c94c4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:42.129576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a7e41-3676-4e41-981d-1b823b8c94c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:42.162516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1f9a7e41-3676-4e41-981d-1b823b8c94c4 None None] MFA Rules not processed for user `ad07006492bb407da09a6b5387237f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:42.164072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1f9a7e41-3676-4e41-981d-1b823b8c94c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:42.203677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:42.283485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1072/2144] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:42.307438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd86d5-e496-4769-963f-a8d0bc4f3dc7 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:42.309748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6dcd86d5-e496-4769-963f-a8d0bc4f3dc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:42.412655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd86d5-e496-4769-963f-a8d0bc4f3dc7 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:42.419851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca2a85-9edd-4dc3-a80a-d3308c48908e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:42.420512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2a85-9edd-4dc3-a80a-d3308c48908e service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:42.420791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2a85-9edd-4dc3-a80a-d3308c48908e service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:42.421064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2a85-9edd-4dc3-a80a-d3308c48908e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:42.422964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-51ca2a85-9edd-4dc3-a80a-d3308c48908e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:42.499349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca2a85-9edd-4dc3-a80a-d3308c48908e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:42.501034 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca2a85-9edd-4dc3-a80a-d3308c48908e service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:42.511932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1073/2145] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:44.357663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:44.526518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a7e41-3676-4e41-981d-1b823b8c94c4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:44.627798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a7e41-3676-4e41-981d-1b823b8c94c4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:44.641463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:44.643179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:44.643684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:44.644269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] PATH_INFO: `/v3/users/1708ab5cf6b24fad9171f343d540698f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:44.645975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1708ab5cf6b24fad9171f343d540698f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:44.648140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:44.879312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1708ab5cf6b24fad9171f343d540698f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:44.916968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1708ab5cf6b24fad9171f343d540698f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:44.936472 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1708ab5cf6b24fad9171f343d540698f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:45.047573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1708ab5cf6b24fad9171f343d540698f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:45.048397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1708ab5cf6b24fad9171f343d540698f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:45.057584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1073/2146] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:44 2022] DELETE /identity/v3/users/1708ab5cf6b24fad9171f343d540698f => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:45.062227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b8540-300f-482a-85d5-8891a3ac6e7c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:45.062724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b8540-300f-482a-85d5-8891a3ac6e7c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:45.062801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b8540-300f-482a-85d5-8891a3ac6e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:45.083980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-1c6b8540-300f-482a-85d5-8891a3ac6e7c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:45.085237 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1c6b8540-300f-482a-85d5-8891a3ac6e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:45.144922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:45.155051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1074/2147] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:45.168997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:45.173938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:45.243643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82d871-9c81-4be3-afed-af8888237e0f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:45.249491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859871a-b9b3-4ebf-a505-a9c853e63be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:45.250484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859871a-b9b3-4ebf-a505-a9c853e63be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:45.250945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859871a-b9b3-4ebf-a505-a9c853e63be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:45.251230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859871a-b9b3-4ebf-a505-a9c853e63be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:45.272653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859871a-b9b3-4ebf-a505-a9c853e63be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:45.274436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859871a-b9b3-4ebf-a505-a9c853e63be1 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:45.285640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1074/2148] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:45.895432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b8540-300f-482a-85d5-8891a3ac6e7c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:45.896865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1c6b8540-300f-482a-85d5-8891a3ac6e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:45.910628 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b8540-300f-482a-85d5-8891a3ac6e7c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:45.915523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:45.916610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:45.917181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:45.918146 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] PATH_INFO: `/v3/projects/e32f7f4be6e3422db1bd5c39b3fac909` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:45.919737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e32f7f4be6e3422db1bd5c39b3fac909)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:45.921887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:46.049908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e32f7f4be6e3422db1bd5c39b3fac909', 'request_id': 'req-d11f8da1-f391-41d4-be78-3fc936608c8c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:46.059425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e32f7f4be6e3422db1bd5c39b3fac909', 'request_id': 'req-d11f8da1-f391-41d4-be78-3fc936608c8c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:46.077184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e32f7f4be6e3422db1bd5c39b3fac909', 'request_id': 'req-d11f8da1-f391-41d4-be78-3fc936608c8c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:46.231194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] The token cache is being invalidate because project e32f7f4be6e3422db1bd5c39b3fac909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:46.231693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:46.233830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d11f8da1-f391-41d4-be78-3fc936608c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:46.236733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1075/2149] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:45 2022] DELETE /identity/v3/projects/e32f7f4be6e3422db1bd5c39b3fac909 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:46.242204 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a64d69-d280-4778-9403-c8c2faa7de5c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:46.242730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a64d69-d280-4778-9403-c8c2faa7de5c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:46.242837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a64d69-d280-4778-9403-c8c2faa7de5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:46.302025 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-78a64d69-d280-4778-9403-c8c2faa7de5c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:46.303433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-78a64d69-d280-4778-9403-c8c2faa7de5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:46.371417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9250b6-f0ee-4789-bb4a-ef6c94652d3e None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:46.372047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9250b6-f0ee-4789-bb4a-ef6c94652d3e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:46.372453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9250b6-f0ee-4789-bb4a-ef6c94652d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:46.380166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1075/2150] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:46.400067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9e9250b6-f0ee-4789-bb4a-ef6c94652d3e None None] MFA Rules not processed for user `d352cf5c00ae450c9a7ca4dcc68a87db`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:46.401353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9e9250b6-f0ee-4789-bb4a-ef6c94652d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:46.407332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d674f7-fa95-4f9d-911a-3ef95621c8fe None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:46.407617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d674f7-fa95-4f9d-911a-3ef95621c8fe None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:46.411361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d674f7-fa95-4f9d-911a-3ef95621c8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:46.467001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b7d674f7-fa95-4f9d-911a-3ef95621c8fe None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:46.468517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b7d674f7-fa95-4f9d-911a-3ef95621c8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:46.478603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1076/2151] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:45:46.502934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1076/2152] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:46.507914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9250b6-f0ee-4789-bb4a-ef6c94652d3e None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:46.526656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9250b6-f0ee-4789-bb4a-ef6c94652d3e None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:46.550314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbc537-1770-4229-ab56-314b918be706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:46.551368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbc537-1770-4229-ab56-314b918be706 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:46.552084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbc537-1770-4229-ab56-314b918be706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:46.552641 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbc537-1770-4229-ab56-314b918be706 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:46.554758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbc537-1770-4229-ab56-314b918be706 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:45:46.555278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbc537-1770-4229-ab56-314b918be706 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:46.558985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbc537-1770-4229-ab56-314b918be706 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:46.605618 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1077/2153] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:45:46.606231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d674f7-fa95-4f9d-911a-3ef95621c8fe None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:46.620987 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d674f7-fa95-4f9d-911a-3ef95621c8fe None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:46.642220 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685623c-bd29-4305-a9d9-1262b1f013b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:46.643280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685623c-bd29-4305-a9d9-1262b1f013b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:46.643756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685623c-bd29-4305-a9d9-1262b1f013b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:46.644236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685623c-bd29-4305-a9d9-1262b1f013b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:46.645129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685623c-bd29-4305-a9d9-1262b1f013b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:46.648073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685623c-bd29-4305-a9d9-1262b1f013b6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:46.798457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbc537-1770-4229-ab56-314b918be706 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:46.809672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1077/2154] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:45:46 2022] POST /identity/v3/projects => generated 371 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:45:46.820457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbc537-1770-4229-ab56-314b918be706 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:46.926440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:46.968733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65fa64-7c69-4b98-b7ff-02fa584f6508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:46.978038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65fa64-7c69-4b98-b7ff-02fa584f6508 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:46.978331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65fa64-7c69-4b98-b7ff-02fa584f6508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:46.978618 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65fa64-7c69-4b98-b7ff-02fa584f6508 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:46.981870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65fa64-7c69-4b98-b7ff-02fa584f6508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:47.001360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65fa64-7c69-4b98-b7ff-02fa584f6508 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:47.156482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1078/2155] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:45:46 2022] POST /identity/v3/users => generated 286 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:45:47.160750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685623c-bd29-4305-a9d9-1262b1f013b6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:47.172384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685623c-bd29-4305-a9d9-1262b1f013b6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:47.178232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fec3b5-0ced-4141-8028-b99e52f16fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:47.178916 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec3b5-0ced-4141-8028-b99e52f16fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:47.179370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec3b5-0ced-4141-8028-b99e52f16fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:47.184873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec3b5-0ced-4141-8028-b99e52f16fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:47.184873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fec3b5-0ced-4141-8028-b99e52f16fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:47.184873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fec3b5-0ced-4141-8028-b99e52f16fdd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:47.196847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1078/2156] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:47.202847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65fa64-7c69-4b98-b7ff-02fa584f6508 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:47.209745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:47.213903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65fa64-7c69-4b98-b7ff-02fa584f6508 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:47.219561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26707165-9eca-4308-a045-7a557e41f9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:47.220357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707165-9eca-4308-a045-7a557e41f9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:47.220717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707165-9eca-4308-a045-7a557e41f9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:47.221251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707165-9eca-4308-a045-7a557e41f9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:47.222238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26707165-9eca-4308-a045-7a557e41f9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:47.224141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26707165-9eca-4308-a045-7a557e41f9d0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:47.240167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1079/2157] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:45:47.241215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fec3b5-0ced-4141-8028-b99e52f16fdd admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:47.254003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fec3b5-0ced-4141-8028-b99e52f16fdd admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:47.261370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aecf8-a15b-4db8-8c4b-a7e4e62b4fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:47.262562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aecf8-a15b-4db8-8c4b-a7e4e62b4fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:47.262890 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aecf8-a15b-4db8-8c4b-a7e4e62b4fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:47.263209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aecf8-a15b-4db8-8c4b-a7e4e62b4fbd admin admin] PATH_INFO: `/v3/projects/63b21ab8b3fa4df6bffdbc4932882f68/users/ec8791a57e274bee96206fad57af0c56/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:47.285621 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99aecf8-a15b-4db8-8c4b-a7e4e62b4fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=63b21ab8b3fa4df6bffdbc4932882f68, user_id=ec8791a57e274bee96206fad57af0c56, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:47.288477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99aecf8-a15b-4db8-8c4b-a7e4e62b4fbd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:47.391184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1079/2158] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:45:47 2022] PUT /identity/v3/projects/63b21ab8b3fa4df6bffdbc4932882f68/users/ec8791a57e274bee96206fad57af0c56/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:47.402454 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f513-26e0-4e9c-b0f1-7f4eb20f5319 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:47.402743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f513-26e0-4e9c-b0f1-7f4eb20f5319 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:47.403352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f513-26e0-4e9c-b0f1-7f4eb20f5319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:47.436526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-13c7f513-26e0-4e9c-b0f1-7f4eb20f5319 None None] MFA Rules not processed for user `ec8791a57e274bee96206fad57af0c56`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:47.437905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-13c7f513-26e0-4e9c-b0f1-7f4eb20f5319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:47.520733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1080/2159] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:47.535836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aecf8-a15b-4db8-8c4b-a7e4e62b4fbd admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:47.538970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c99aecf8-a15b-4db8-8c4b-a7e4e62b4fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:47.614659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aecf8-a15b-4db8-8c4b-a7e4e62b4fbd admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:47.618634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd64fdd-c351-4ad9-881c-6fcfbefbf3be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1a7213cbf6448b7ba6570a52a4c7c0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:47.620264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd64fdd-c351-4ad9-881c-6fcfbefbf3be service glance] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:47.621493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd64fdd-c351-4ad9-881c-6fcfbefbf3be service glance] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:47.621769 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd64fdd-c351-4ad9-881c-6fcfbefbf3be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:47.633561 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd64fdd-c351-4ad9-881c-6fcfbefbf3be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:47.635245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd64fdd-c351-4ad9-881c-6fcfbefbf3be service glance] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:47.647709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1080/2160] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:47.968151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:49.868810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7f513-26e0-4e9c-b0f1-7f4eb20f5319 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:49.871150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-13c7f513-26e0-4e9c-b0f1-7f4eb20f5319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:49.959518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7f513-26e0-4e9c-b0f1-7f4eb20f5319 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:49.964662 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:49.965579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:49.965862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:49.966138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] PATH_INFO: `/v3/users/a5147f8dd28a4ababbc4886861f2b6f6` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:49.967348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5147f8dd28a4ababbc4886861f2b6f6)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:49.969514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:50.179842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5147f8dd28a4ababbc4886861f2b6f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:50.180538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5147f8dd28a4ababbc4886861f2b6f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:50.194462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5147f8dd28a4ababbc4886861f2b6f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:50.201881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5147f8dd28a4ababbc4886861f2b6f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:50.291332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5147f8dd28a4ababbc4886861f2b6f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:50.299361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1081/2161] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:45:49 2022] DELETE /identity/v3/users/a5147f8dd28a4ababbc4886861f2b6f6 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:50.304238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba007bd8-fc10-4da0-a213-ad185627e99b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:50.304855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba007bd8-fc10-4da0-a213-ad185627e99b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:50.304855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba007bd8-fc10-4da0-a213-ad185627e99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:50.323544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ba007bd8-fc10-4da0-a213-ad185627e99b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:50.324815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ba007bd8-fc10-4da0-a213-ad185627e99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:50.399473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1081/2162] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:45:50.409820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:50.411352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:50.417225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:50.473261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25527b51-812a-4de8-b89f-40f45275a4ef admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:50.481364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85717be-0381-4d9d-8470-db526cce51e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:50.483492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85717be-0381-4d9d-8470-db526cce51e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:50.483492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85717be-0381-4d9d-8470-db526cce51e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:50.483492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85717be-0381-4d9d-8470-db526cce51e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:50.493227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85717be-0381-4d9d-8470-db526cce51e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:50.494697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85717be-0381-4d9d-8470-db526cce51e4 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:50.505246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1082/2163] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:45:51.061733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba007bd8-fc10-4da0-a213-ad185627e99b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:45:51.063209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ba007bd8-fc10-4da0-a213-ad185627e99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:51.075102 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba007bd8-fc10-4da0-a213-ad185627e99b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:45:51.085461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:45:51.085461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:51.085461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:51.085461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] PATH_INFO: `/v3/projects/1ef60975c92640fe9b7f96d5f6236b8d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:51.085777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ef60975c92640fe9b7f96d5f6236b8d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:45:51.086522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:45:51.219279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ef60975c92640fe9b7f96d5f6236b8d', 'request_id': 'req-392aac5b-e20b-42a2-a223-8eb89ce8af92'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:51.299379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ef60975c92640fe9b7f96d5f6236b8d', 'request_id': 'req-392aac5b-e20b-42a2-a223-8eb89ce8af92'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:51.301092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:45:51.316983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ef60975c92640fe9b7f96d5f6236b8d', 'request_id': 'req-392aac5b-e20b-42a2-a223-8eb89ce8af92'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:51.318869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] The token cache is being invalidate because project 1ef60975c92640fe9b7f96d5f6236b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:45:51.319155 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:51.319808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-392aac5b-e20b-42a2-a223-8eb89ce8af92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:45:51.321417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1082/2164] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:45:51 2022] DELETE /identity/v3/projects/1ef60975c92640fe9b7f96d5f6236b8d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:45:54.206442 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebfa67-9353-41f1-b698-fb0de73d8e9b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:45:54.206936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebfa67-9353-41f1-b698-fb0de73d8e9b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:45:54.207044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebfa67-9353-41f1-b698-fb0de73d8e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:45:54.226086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-37ebfa67-9353-41f1-b698-fb0de73d8e9b None None] MFA Rules not processed for user `8773c88f88614a9d903f667305a217e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:45:54.227288 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-37ebfa67-9353-41f1-b698-fb0de73d8e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:45:54.284881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1083/2165] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:45:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:46:03.086299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=2,keystone:SELECT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:04.277368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=50 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:09.766356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff44630-fadf-4154-b556-505e2d73fd6d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:09.767123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff44630-fadf-4154-b556-505e2d73fd6d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:09.767123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff44630-fadf-4154-b556-505e2d73fd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:09.786024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-dff44630-fadf-4154-b556-505e2d73fd6d None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:46:09.787471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dff44630-fadf-4154-b556-505e2d73fd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:09.858818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1083/2166] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:46:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:46:09.908398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebfa67-9353-41f1-b698-fb0de73d8e9b None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:09.908398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-37ebfa67-9353-41f1-b698-fb0de73d8e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:09.984549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebfa67-9353-41f1-b698-fb0de73d8e9b None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:09.991314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f29415-5303-4f8a-9b72-61db7bc461bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:09.991985 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f29415-5303-4f8a-9b72-61db7bc461bd service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:09.992272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f29415-5303-4f8a-9b72-61db7bc461bd service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:09.993232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f29415-5303-4f8a-9b72-61db7bc461bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:10.009815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f29415-5303-4f8a-9b72-61db7bc461bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:10.011966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f29415-5303-4f8a-9b72-61db7bc461bd service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:10.021187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1084/2167] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:46:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:46:19.851074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=25 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:20.017867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=32 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:28.738921 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff44630-fadf-4154-b556-505e2d73fd6d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:28.740378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dff44630-fadf-4154-b556-505e2d73fd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:28.816596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff44630-fadf-4154-b556-505e2d73fd6d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:28.821198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:28.822067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:28.822349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:28.822625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] PATH_INFO: `/v3/users/ec8791a57e274bee96206fad57af0c56` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:28.823833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec8791a57e274bee96206fad57af0c56)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:28.826336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:29.084237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec8791a57e274bee96206fad57af0c56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:46:29.090976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec8791a57e274bee96206fad57af0c56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:46:29.102945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec8791a57e274bee96206fad57af0c56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:46:29.165750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec8791a57e274bee96206fad57af0c56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:46:29.166399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec8791a57e274bee96206fad57af0c56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:46:29.172965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1084/2168] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:46:28 2022] DELETE /identity/v3/users/ec8791a57e274bee96206fad57af0c56 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:46:29.177531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5b940-3a9b-43ef-9416-898843ac1909 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:29.178040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5b940-3a9b-43ef-9416-898843ac1909 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:29.178144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5b940-3a9b-43ef-9416-898843ac1909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:29.195975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-89a5b940-3a9b-43ef-9416-898843ac1909 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:46:29.197318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-89a5b940-3a9b-43ef-9416-898843ac1909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:29.255003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1085/2169] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:46:29.265249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:29.266655 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:29.328477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554b9dc-c1e4-4e83-85cf-b009a3fd7d18 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:29.332871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182389a-3ca3-4cfc-a703-a89964d242e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:29.333563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182389a-3ca3-4cfc-a703-a89964d242e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:29.333846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182389a-3ca3-4cfc-a703-a89964d242e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:29.334122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182389a-3ca3-4cfc-a703-a89964d242e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:29.343466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182389a-3ca3-4cfc-a703-a89964d242e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:29.344836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182389a-3ca3-4cfc-a703-a89964d242e9 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:29.354699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1085/2170] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:46:29.864159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5b940-3a9b-43ef-9416-898843ac1909 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:29.866111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-89a5b940-3a9b-43ef-9416-898843ac1909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:29.880197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5b940-3a9b-43ef-9416-898843ac1909 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:29.884933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:29.885874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:29.886187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:29.886476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] PATH_INFO: `/v3/projects/63b21ab8b3fa4df6bffdbc4932882f68` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:29.888314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63b21ab8b3fa4df6bffdbc4932882f68)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:29.890414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:30.048630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63b21ab8b3fa4df6bffdbc4932882f68', 'request_id': 'req-c3e4ad9b-8fac-4743-8dad-a55395321bf6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:46:30.056384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63b21ab8b3fa4df6bffdbc4932882f68', 'request_id': 'req-c3e4ad9b-8fac-4743-8dad-a55395321bf6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:46:30.063559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63b21ab8b3fa4df6bffdbc4932882f68', 'request_id': 'req-c3e4ad9b-8fac-4743-8dad-a55395321bf6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:46:30.148538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] The token cache is being invalidate because project 63b21ab8b3fa4df6bffdbc4932882f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:46:30.148832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:46:30.150941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-c3e4ad9b-8fac-4743-8dad-a55395321bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:46:30.151282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1086/2171] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:46:29 2022] DELETE /identity/v3/projects/63b21ab8b3fa4df6bffdbc4932882f68 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:46:30.156307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da85bd-a115-4141-934e-9578010780db None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:30.156826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da85bd-a115-4141-934e-9578010780db None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:30.157109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da85bd-a115-4141-934e-9578010780db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:30.177501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e3da85bd-a115-4141-934e-9578010780db None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:46:30.179226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e3da85bd-a115-4141-934e-9578010780db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:30.201576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:30.250851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1086/2172] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:46:30.259433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e2547-c6d6-4384-9532-e151b19d9b0e None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:30.259968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e2547-c6d6-4384-9532-e151b19d9b0e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:30.259968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e2547-c6d6-4384-9532-e151b19d9b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:30.279829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-cd1e2547-c6d6-4384-9532-e151b19d9b0e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:46:30.283677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cd1e2547-c6d6-4384-9532-e151b19d9b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:30.290313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1087/2173] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:46:30.294634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da85bd-a115-4141-934e-9578010780db None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:30.306573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da85bd-a115-4141-934e-9578010780db None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:30.314324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a70904-6273-4f01-b85f-a12af3315df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:30.318072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a70904-6273-4f01-b85f-a12af3315df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:30.318072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a70904-6273-4f01-b85f-a12af3315df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:30.318072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a70904-6273-4f01-b85f-a12af3315df8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:30.318072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a70904-6273-4f01-b85f-a12af3315df8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:46:30.318072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a70904-6273-4f01-b85f-a12af3315df8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:30.319090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a70904-6273-4f01-b85f-a12af3315df8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:30.338840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1087/2174] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:46:30.340499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e2547-c6d6-4384-9532-e151b19d9b0e None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:30.351053 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e2547-c6d6-4384-9532-e151b19d9b0e None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:30.356570 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166c7e2-ddd6-40e5-8ea3-789a7c1f4ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:30.357292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166c7e2-ddd6-40e5-8ea3-789a7c1f4ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:30.357573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166c7e2-ddd6-40e5-8ea3-789a7c1f4ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:30.358003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166c7e2-ddd6-40e5-8ea3-789a7c1f4ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:30.358726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166c7e2-ddd6-40e5-8ea3-789a7c1f4ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:30.360676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166c7e2-ddd6-40e5-8ea3-789a7c1f4ca9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:30.462166 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1088/2175] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:46:30 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:46:30.464884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a70904-6273-4f01-b85f-a12af3315df8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:30.475597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a70904-6273-4f01-b85f-a12af3315df8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:30.535226 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed833c2c-8def-47a3-8aa4-9a89b28c0614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:30.535943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed833c2c-8def-47a3-8aa4-9a89b28c0614 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:30.536244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed833c2c-8def-47a3-8aa4-9a89b28c0614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:30.536527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed833c2c-8def-47a3-8aa4-9a89b28c0614 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:30.537262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed833c2c-8def-47a3-8aa4-9a89b28c0614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:30.539179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed833c2c-8def-47a3-8aa4-9a89b28c0614 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:30.648658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1088/2176] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:46:30 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:46:30.653397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166c7e2-ddd6-40e5-8ea3-789a7c1f4ca9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:30.663853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166c7e2-ddd6-40e5-8ea3-789a7c1f4ca9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:30.669180 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dee943-f5c9-4171-9672-03b016a09476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:30.669996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dee943-f5c9-4171-9672-03b016a09476 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:30.670281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dee943-f5c9-4171-9672-03b016a09476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:30.670556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dee943-f5c9-4171-9672-03b016a09476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:30.674216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dee943-f5c9-4171-9672-03b016a09476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:30.674216 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dee943-f5c9-4171-9672-03b016a09476 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:30.684392 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1089/2177] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:46:30.688376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed833c2c-8def-47a3-8aa4-9a89b28c0614 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:30.698656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed833c2c-8def-47a3-8aa4-9a89b28c0614 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:30.703449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc2049-0240-44df-9a6d-a8fd96b7fd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:30.704123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc2049-0240-44df-9a6d-a8fd96b7fd80 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:30.704403 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc2049-0240-44df-9a6d-a8fd96b7fd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:30.704675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc2049-0240-44df-9a6d-a8fd96b7fd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:30.705434 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc2049-0240-44df-9a6d-a8fd96b7fd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:30.706999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc2049-0240-44df-9a6d-a8fd96b7fd80 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:30.717838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1089/2178] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:46:30.721867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dee943-f5c9-4171-9672-03b016a09476 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:30.732546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dee943-f5c9-4171-9672-03b016a09476 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:30.737416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ece3e-95e5-4c39-8488-ee737d398f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:30.738304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ece3e-95e5-4c39-8488-ee737d398f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:30.738590 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ece3e-95e5-4c39-8488-ee737d398f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:30.738893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ece3e-95e5-4c39-8488-ee737d398f23 admin admin] PATH_INFO: `/v3/projects/1ec44d80054e49e9ba17a711740b1a3d/users/983f6117721d436786fe8827a13a3f01/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:30.754145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ece3e-95e5-4c39-8488-ee737d398f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec44d80054e49e9ba17a711740b1a3d, user_id=983f6117721d436786fe8827a13a3f01, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:30.760270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ece3e-95e5-4c39-8488-ee737d398f23 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:30.843098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1090/2179] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:46:30 2022] PUT /identity/v3/projects/1ec44d80054e49e9ba17a711740b1a3d/users/983f6117721d436786fe8827a13a3f01/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:46:30.853440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc425dba-ab7c-4ed0-a773-29b634242af6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:30.853740 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc425dba-ab7c-4ed0-a773-29b634242af6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:30.854021 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc425dba-ab7c-4ed0-a773-29b634242af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:30.882334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fc425dba-ab7c-4ed0-a773-29b634242af6 None None] MFA Rules not processed for user `983f6117721d436786fe8827a13a3f01`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:46:30.883537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fc425dba-ab7c-4ed0-a773-29b634242af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:30.947706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1090/2180] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:46:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:46:30.959063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ece3e-95e5-4c39-8488-ee737d398f23 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:30.960806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3a4ece3e-95e5-4c39-8488-ee737d398f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:30.983321 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:31.019802 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ece3e-95e5-4c39-8488-ee737d398f23 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:31.023523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00991241-6af3-440b-b3b7-7889be5ded9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:31.024436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00991241-6af3-440b-b3b7-7889be5ded9c service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:31.029071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00991241-6af3-440b-b3b7-7889be5ded9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:31.029071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00991241-6af3-440b-b3b7-7889be5ded9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:31.036635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00991241-6af3-440b-b3b7-7889be5ded9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:31.038186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00991241-6af3-440b-b3b7-7889be5ded9c service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:31.047796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1091/2181] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:46:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:46:31.519876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9170-3564-43e3-b809-60bf06a868a1 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:31.520889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9170-3564-43e3-b809-60bf06a868a1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:31.521246 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9170-3564-43e3-b809-60bf06a868a1 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:31.522782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1091/2182] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:46:31.529376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91d8c-9c68-474d-81d4-7491e0dc3ca4 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:31.530253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91d8c-9c68-474d-81d4-7491e0dc3ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:31.530871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91d8c-9c68-474d-81d4-7491e0dc3ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:31.551934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-dcf91d8c-9c68-474d-81d4-7491e0dc3ca4 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:46:31.553305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-dcf91d8c-9c68-474d-81d4-7491e0dc3ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:31.644123 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1092/2183] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:46:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:46:31.658288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e9170-3564-43e3-b809-60bf06a868a1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:31.664105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0f8e9170-3564-43e3-b809-60bf06a868a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:31.726317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:31.773404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e9170-3564-43e3-b809-60bf06a868a1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:31.777231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc1d59-8701-47e0-8768-d27aa58d41c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1a7213cbf6448b7ba6570a52a4c7c0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:31.778454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc1d59-8701-47e0-8768-d27aa58d41c6 service glance] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:31.778870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc1d59-8701-47e0-8768-d27aa58d41c6 service glance] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:31.779163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc1d59-8701-47e0-8768-d27aa58d41c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:31.789851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc1d59-8701-47e0-8768-d27aa58d41c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:31.791369 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc1d59-8701-47e0-8768-d27aa58d41c6 service glance] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:31.804813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1092/2184] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:46:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:46:31.991000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f497a-7fa6-46d6-980e-cc5187ab202f None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:31.991289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f497a-7fa6-46d6-980e-cc5187ab202f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:31.992509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f497a-7fa6-46d6-980e-cc5187ab202f None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:31.992933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1093/2185] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:46:31.998758 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782148c-7f7b-4e91-8a2c-9d9648330481 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:31.998984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782148c-7f7b-4e91-8a2c-9d9648330481 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:31.999286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782148c-7f7b-4e91-8a2c-9d9648330481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:32.018003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8782148c-7f7b-4e91-8a2c-9d9648330481 None None] MFA Rules not processed for user `481ef56b1cfa465a8efc59736a34e412`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:46:32.019233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8782148c-7f7b-4e91-8a2c-9d9648330481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:32.072248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1093/2186] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:46:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:46:32.084838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f497a-7fa6-46d6-980e-cc5187ab202f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:32.086594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8a3f497a-7fa6-46d6-980e-cc5187ab202f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:32.099619 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f497a-7fa6-46d6-980e-cc5187ab202f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:32.103389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a743d-f74f-4fa1-b0db-199511447c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:32.104064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a743d-f74f-4fa1-b0db-199511447c9a service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:32.104352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a743d-f74f-4fa1-b0db-199511447c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:32.104626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a743d-f74f-4fa1-b0db-199511447c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:32.112266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a743d-f74f-4fa1-b0db-199511447c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:32.113750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a743d-f74f-4fa1-b0db-199511447c9a service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:32.122273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1094/2187] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:46:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:46:32.245864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ccf81-eca8-41f1-b675-c45fd4558b5d None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:32.246148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ccf81-eca8-41f1-b675-c45fd4558b5d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:32.246420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ccf81-eca8-41f1-b675-c45fd4558b5d None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:32.248689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1094/2188] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:46:32.254394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dbd18-5f94-4ab5-8db2-ce3874fcb7d6 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:32.254690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dbd18-5f94-4ab5-8db2-ce3874fcb7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:32.255023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dbd18-5f94-4ab5-8db2-ce3874fcb7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:32.275725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-876dbd18-5f94-4ab5-8db2-ce3874fcb7d6 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:46:32.276933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-876dbd18-5f94-4ab5-8db2-ce3874fcb7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:32.286352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1095/2189] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:46:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:46:32.297394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ccf81-eca8-41f1-b675-c45fd4558b5d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:32.310355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ccf81-eca8-41f1-b675-c45fd4558b5d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:32.314146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3230ad6-cab9-4b10-a23a-1d3172bb14d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:32.314953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3230ad6-cab9-4b10-a23a-1d3172bb14d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:32.315259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3230ad6-cab9-4b10-a23a-1d3172bb14d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:32.315635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3230ad6-cab9-4b10-a23a-1d3172bb14d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:32.329410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3230ad6-cab9-4b10-a23a-1d3172bb14d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:32.331262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3230ad6-cab9-4b10-a23a-1d3172bb14d6 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:32.348703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1095/2190] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:46:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:46:32.819766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70493d61-3de3-4555-bad3-853b9e8ede01 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:32.820700 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70493d61-3de3-4555-bad3-853b9e8ede01 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:32.821223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70493d61-3de3-4555-bad3-853b9e8ede01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:32.840054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-70493d61-3de3-4555-bad3-853b9e8ede01 None None] MFA Rules not processed for user `983f6117721d436786fe8827a13a3f01`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:46:32.841282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-70493d61-3de3-4555-bad3-853b9e8ede01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:32.849180 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1096/2191] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:46:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:46:42.345214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=47 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:42.834808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=60 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:45.726212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10154b5f-e49c-453f-a93b-99ecabc1e4f9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:45.730208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10154b5f-e49c-453f-a93b-99ecabc1e4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:45.730208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10154b5f-e49c-453f-a93b-99ecabc1e4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:45.749684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-10154b5f-e49c-453f-a93b-99ecabc1e4f9 None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:46:45.751041 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-10154b5f-e49c-453f-a93b-99ecabc1e4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:45.862929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1096/2192] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:46:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:46:45.879532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70493d61-3de3-4555-bad3-853b9e8ede01 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:45.882666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-70493d61-3de3-4555-bad3-853b9e8ede01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:45.978419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70493d61-3de3-4555-bad3-853b9e8ede01 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:45.989233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e464f-9e5b-4093-9c9e-9c6702d63dea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:45.989233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e464f-9e5b-4093-9c9e-9c6702d63dea service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:45.989233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e464f-9e5b-4093-9c9e-9c6702d63dea service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:45.989233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e464f-9e5b-4093-9c9e-9c6702d63dea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:46.001755 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e464f-9e5b-4093-9c9e-9c6702d63dea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:46.003367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e464f-9e5b-4093-9c9e-9c6702d63dea service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:46.013230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1097/2193] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:46:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:46:55.855525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=25 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:56.009281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=32 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:46:58.147230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10154b5f-e49c-453f-a93b-99ecabc1e4f9 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:58.157626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10154b5f-e49c-453f-a93b-99ecabc1e4f9 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:58.161329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a195f8-7415-4e1e-bc9f-abb5a2c119c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:58.161981 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a195f8-7415-4e1e-bc9f-abb5a2c119c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:58.162265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a195f8-7415-4e1e-bc9f-abb5a2c119c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:58.162540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a195f8-7415-4e1e-bc9f-abb5a2c119c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:58.164420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-90a195f8-7415-4e1e-bc9f-abb5a2c119c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:58.234433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a195f8-7415-4e1e-bc9f-abb5a2c119c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:58.235882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a195f8-7415-4e1e-bc9f-abb5a2c119c7 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:58.245988 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1097/2194] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:46:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:46:59.698261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e464f-9e5b-4093-9c9e-9c6702d63dea service swift] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:46:59.700266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cf0e464f-9e5b-4093-9c9e-9c6702d63dea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:59.781063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e464f-9e5b-4093-9c9e-9c6702d63dea service swift] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:46:59.788011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91c633-bfd5-45a9-a8d6-08a249ed4f8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:46:59.789076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91c633-bfd5-45a9-a8d6-08a249ed4f8e service placement] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:46:59.789400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91c633-bfd5-45a9-a8d6-08a249ed4f8e service placement] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:46:59.789678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91c633-bfd5-45a9-a8d6-08a249ed4f8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:46:59.791747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-db91c633-bfd5-45a9-a8d6-08a249ed4f8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:46:59.806461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91c633-bfd5-45a9-a8d6-08a249ed4f8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:46:59.808080 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91c633-bfd5-45a9-a8d6-08a249ed4f8e service placement] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:46:59.820057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1098/2195] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:46:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:47:08.241592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=30 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:47:09.815526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=32 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:47:15.666228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034db97-4d64-43f5-9332-60cf97eeef52 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:15.666725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034db97-4d64-43f5-9332-60cf97eeef52 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:15.666834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034db97-4d64-43f5-9332-60cf97eeef52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:15.688479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6034db97-4d64-43f5-9332-60cf97eeef52 None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:15.688637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6034db97-4d64-43f5-9332-60cf97eeef52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:15.702676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1098/2196] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jul 14 11:47:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:15.726977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91c633-bfd5-45a9-a8d6-08a249ed4f8e service placement] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:15.750022 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91c633-bfd5-45a9-a8d6-08a249ed4f8e service placement] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:15.756090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e813244-1226-440d-8cab-57dbab39dd5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:15.756853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e813244-1226-440d-8cab-57dbab39dd5e service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:15.757278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e813244-1226-440d-8cab-57dbab39dd5e service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:15.757644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e813244-1226-440d-8cab-57dbab39dd5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:15.768676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e813244-1226-440d-8cab-57dbab39dd5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:15.770488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e813244-1226-440d-8cab-57dbab39dd5e service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:15.781960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1099/2197] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:47:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:47:17.146893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabb75b-38a8-4d78-bc4b-07b6301c82d4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:17.147808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabb75b-38a8-4d78-bc4b-07b6301c82d4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:17.148669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabb75b-38a8-4d78-bc4b-07b6301c82d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:17.168583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-adabb75b-38a8-4d78-bc4b-07b6301c82d4 None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:17.170651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-adabb75b-38a8-4d78-bc4b-07b6301c82d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:17.179773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1099/2198] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:47:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:17.195505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e813244-1226-440d-8cab-57dbab39dd5e service swift] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:17.208218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e813244-1226-440d-8cab-57dbab39dd5e service swift] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:17.213011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2c29a-7564-4003-827c-c4febad257d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:17.213994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2c29a-7564-4003-827c-c4febad257d1 service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:17.214278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2c29a-7564-4003-827c-c4febad257d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:17.214728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2c29a-7564-4003-827c-c4febad257d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:17.223912 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2c29a-7564-4003-827c-c4febad257d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:17.225562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2c29a-7564-4003-827c-c4febad257d1 service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:17.234642 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1100/2199] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:47:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:47:18.050658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2c5bd-3811-4f35-91d2-1c3b0368208e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:18.050952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2c5bd-3811-4f35-91d2-1c3b0368208e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:18.051234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2c5bd-3811-4f35-91d2-1c3b0368208e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:18.070002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b4a2c5bd-3811-4f35-91d2-1c3b0368208e None None] MFA Rules not processed for user `7bf3f705e04640a2a5d9bedb71c6ccb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:18.071425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b4a2c5bd-3811-4f35-91d2-1c3b0368208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:18.134366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1100/2200] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:18.350062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece346eb-94b6-4f03-abd3-f5c83c120084 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:18.350583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece346eb-94b6-4f03-abd3-f5c83c120084 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:18.350682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece346eb-94b6-4f03-abd3-f5c83c120084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:18.369187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ece346eb-94b6-4f03-abd3-f5c83c120084 None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:18.370583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ece346eb-94b6-4f03-abd3-f5c83c120084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:18.378750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1101/2201] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jul 14 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:18.391891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2c5bd-3811-4f35-91d2-1c3b0368208e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:18.405128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2c5bd-3811-4f35-91d2-1c3b0368208e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:18.410295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf56971-a804-4b8e-be9d-2b6f790050da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:18.411091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf56971-a804-4b8e-be9d-2b6f790050da service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:18.411287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf56971-a804-4b8e-be9d-2b6f790050da service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:18.411736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf56971-a804-4b8e-be9d-2b6f790050da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:18.421009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf56971-a804-4b8e-be9d-2b6f790050da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:18.423191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf56971-a804-4b8e-be9d-2b6f790050da service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:18.432673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1101/2202] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:47:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:47:19.105796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24b3cf-4256-436f-bd41-7ee888061641 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:19.105796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24b3cf-4256-436f-bd41-7ee888061641 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:19.106550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24b3cf-4256-436f-bd41-7ee888061641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:19.131127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-ea24b3cf-4256-436f-bd41-7ee888061641 None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:19.132956 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ea24b3cf-4256-436f-bd41-7ee888061641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:19.141246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1102/2203] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:47:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:19.153074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf56971-a804-4b8e-be9d-2b6f790050da service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:19.163612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf56971-a804-4b8e-be9d-2b6f790050da service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:19.169099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147baa8-f5b8-4208-b872-dcda4f15ec26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:19.170170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147baa8-f5b8-4208-b872-dcda4f15ec26 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:19.170523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147baa8-f5b8-4208-b872-dcda4f15ec26 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:19.170876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147baa8-f5b8-4208-b872-dcda4f15ec26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:19.179672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147baa8-f5b8-4208-b872-dcda4f15ec26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:19.181385 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147baa8-f5b8-4208-b872-dcda4f15ec26 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:19.191504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1102/2204] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:47:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:47:19.949124 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75b709-a5f1-4bfc-bb70-0c5469d8e71c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:19.953560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75b709-a5f1-4bfc-bb70-0c5469d8e71c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:19.953560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75b709-a5f1-4bfc-bb70-0c5469d8e71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:19.969183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-2a75b709-a5f1-4bfc-bb70-0c5469d8e71c None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:19.970488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2a75b709-a5f1-4bfc-bb70-0c5469d8e71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:19.977882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1103/2205] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jul 14 11:47:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:19.991878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147baa8-f5b8-4208-b872-dcda4f15ec26 service swift] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:20.005260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147baa8-f5b8-4208-b872-dcda4f15ec26 service swift] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:20.012202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa82e0-6a0f-44a8-ad81-4cfa60c943a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:20.012861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa82e0-6a0f-44a8-ad81-4cfa60c943a4 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:20.017675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa82e0-6a0f-44a8-ad81-4cfa60c943a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:20.018118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa82e0-6a0f-44a8-ad81-4cfa60c943a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:20.032190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa82e0-6a0f-44a8-ad81-4cfa60c943a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:20.033941 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa82e0-6a0f-44a8-ad81-4cfa60c943a4 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:20.042637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1103/2206] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:47:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:47:22.885613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75b709-a5f1-4bfc-bb70-0c5469d8e71c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:22.984399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75b709-a5f1-4bfc-bb70-0c5469d8e71c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:22.995836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:22.996975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:22.997443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:22.997897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] PATH_INFO: `/v3/users/983f6117721d436786fe8827a13a3f01` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:22.999349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=983f6117721d436786fe8827a13a3f01)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:23.001719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:23.331980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983f6117721d436786fe8827a13a3f01'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:47:23.332654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983f6117721d436786fe8827a13a3f01'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:47:23.365647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983f6117721d436786fe8827a13a3f01'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:47:23.381345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983f6117721d436786fe8827a13a3f01'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:47:23.549908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983f6117721d436786fe8827a13a3f01'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:47:23.558549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1104/2207] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:47:22 2022] DELETE /identity/v3/users/983f6117721d436786fe8827a13a3f01 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:47:23.563723 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3801daf-ffc4-414c-aa37-6dd8d1a8e9c0 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:23.564546 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3801daf-ffc4-414c-aa37-6dd8d1a8e9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:23.566266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3801daf-ffc4-414c-aa37-6dd8d1a8e9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:23.596774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a3801daf-ffc4-414c-aa37-6dd8d1a8e9c0 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:23.598101 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a3801daf-ffc4-414c-aa37-6dd8d1a8e9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:23.683239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1104/2208] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:47:23.697144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:23.698766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:23.743448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:47:23.772179 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b13064-779f-4b36-857b-a9d30d2fd023 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:23.777873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c70f7-d58b-494d-bba5-7cd64b2a46cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:23.778631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c70f7-d58b-494d-bba5-7cd64b2a46cd service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:23.779008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c70f7-d58b-494d-bba5-7cd64b2a46cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:23.779292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c70f7-d58b-494d-bba5-7cd64b2a46cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:23.791635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c70f7-d58b-494d-bba5-7cd64b2a46cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:23.793236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c70f7-d58b-494d-bba5-7cd64b2a46cd service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:23.803740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1105/2209] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:47:24.291279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3801daf-ffc4-414c-aa37-6dd8d1a8e9c0 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:24.291279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a3801daf-ffc4-414c-aa37-6dd8d1a8e9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:24.307710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3801daf-ffc4-414c-aa37-6dd8d1a8e9c0 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:24.314436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:24.315573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:24.316059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:24.320037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] PATH_INFO: `/v3/projects/1ec44d80054e49e9ba17a711740b1a3d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:24.320037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ec44d80054e49e9ba17a711740b1a3d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:24.320478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:24.462850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:47:24.464984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ec44d80054e49e9ba17a711740b1a3d', 'request_id': 'req-b139bc3e-f067-4f61-8086-a1ee102af410'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:47:24.527325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ec44d80054e49e9ba17a711740b1a3d', 'request_id': 'req-b139bc3e-f067-4f61-8086-a1ee102af410'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:47:24.536621 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ec44d80054e49e9ba17a711740b1a3d', 'request_id': 'req-b139bc3e-f067-4f61-8086-a1ee102af410'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:47:24.548887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] The token cache is being invalidate because project 1ec44d80054e49e9ba17a711740b1a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:47:24.549482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:47:24.550305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b139bc3e-f067-4f61-8086-a1ee102af410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:47:24.552052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1105/2210] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:47:24 2022] DELETE /identity/v3/projects/1ec44d80054e49e9ba17a711740b1a3d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:47:24.557094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb681a-d0c8-4e02-88b2-bcf7715618b7 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:24.557643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb681a-d0c8-4e02-88b2-bcf7715618b7 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:24.557747 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb681a-d0c8-4e02-88b2-bcf7715618b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:24.585099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e4cb681a-d0c8-4e02-88b2-bcf7715618b7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:24.587050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e4cb681a-d0c8-4e02-88b2-bcf7715618b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:24.675267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1106/2211] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:24.683239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddacb6-d40d-4864-82be-696bf6575145 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:24.683720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddacb6-d40d-4864-82be-696bf6575145 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:24.683852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddacb6-d40d-4864-82be-696bf6575145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:24.703624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-97ddacb6-d40d-4864-82be-696bf6575145 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:24.704732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-97ddacb6-d40d-4864-82be-696bf6575145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:24.713675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1106/2212] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:24.720652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb681a-d0c8-4e02-88b2-bcf7715618b7 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:24.731280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb681a-d0c8-4e02-88b2-bcf7715618b7 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:24.738760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0b274-1593-4e9d-91c5-f94c3805b305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:24.739663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0b274-1593-4e9d-91c5-f94c3805b305 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:24.740552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0b274-1593-4e9d-91c5-f94c3805b305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:24.740552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0b274-1593-4e9d-91c5-f94c3805b305 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:24.740991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0b274-1593-4e9d-91c5-f94c3805b305 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:47:24.741348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0b274-1593-4e9d-91c5-f94c3805b305 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:24.742986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0b274-1593-4e9d-91c5-f94c3805b305 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:24.758804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1107/2213] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:47:24.760838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddacb6-d40d-4864-82be-696bf6575145 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:24.774464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddacb6-d40d-4864-82be-696bf6575145 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:24.784537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74569be3-8161-445b-97e2-c15342ea0ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:24.784934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569be3-8161-445b-97e2-c15342ea0ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:24.792155 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569be3-8161-445b-97e2-c15342ea0ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:24.792511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569be3-8161-445b-97e2-c15342ea0ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:24.800701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74569be3-8161-445b-97e2-c15342ea0ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:24.803537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74569be3-8161-445b-97e2-c15342ea0ebb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:24.905142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1107/2214] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:47:24 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:47:24.907512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0b274-1593-4e9d-91c5-f94c3805b305 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:24.918956 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0b274-1593-4e9d-91c5-f94c3805b305 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:24.987354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e638-e484-4409-9187-5dff564c5cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:24.987354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e638-e484-4409-9187-5dff564c5cae admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:24.987354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e638-e484-4409-9187-5dff564c5cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:24.987354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e638-e484-4409-9187-5dff564c5cae admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:24.987756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389e638-e484-4409-9187-5dff564c5cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:24.987756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389e638-e484-4409-9187-5dff564c5cae admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:25.070588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1108/2215] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:47:24 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:47:25.074837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74569be3-8161-445b-97e2-c15342ea0ebb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:25.086175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74569be3-8161-445b-97e2-c15342ea0ebb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:25.090889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53aa0e1-1d4d-4377-83aa-bc69e0ba4a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:25.091687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aa0e1-1d4d-4377-83aa-bc69e0ba4a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:25.092046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aa0e1-1d4d-4377-83aa-bc69e0ba4a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:25.092395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aa0e1-1d4d-4377-83aa-bc69e0ba4a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:25.093237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53aa0e1-1d4d-4377-83aa-bc69e0ba4a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:25.094894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53aa0e1-1d4d-4377-83aa-bc69e0ba4a11 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:25.108967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1108/2216] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:47:25.110006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e638-e484-4409-9187-5dff564c5cae admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:25.121653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e638-e484-4409-9187-5dff564c5cae admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:25.128085 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e853e06-4976-4184-91a9-67a2b9edc831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:25.129016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e853e06-4976-4184-91a9-67a2b9edc831 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:25.129527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e853e06-4976-4184-91a9-67a2b9edc831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:25.129900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e853e06-4976-4184-91a9-67a2b9edc831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:25.130756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e853e06-4976-4184-91a9-67a2b9edc831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:25.178160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e853e06-4976-4184-91a9-67a2b9edc831 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:25.178160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1109/2217] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:47:25.180567 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53aa0e1-1d4d-4377-83aa-bc69e0ba4a11 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:25.192690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53aa0e1-1d4d-4377-83aa-bc69e0ba4a11 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:25.198844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8594db-437b-409e-8c2c-4f4ad5a539ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:25.200073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8594db-437b-409e-8c2c-4f4ad5a539ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:25.200679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8594db-437b-409e-8c2c-4f4ad5a539ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:25.201190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8594db-437b-409e-8c2c-4f4ad5a539ed admin admin] PATH_INFO: `/v3/projects/dc22323158714269b0da2b01121fe3ed/users/59ebb8f1e98d40c9aea425fad464f1e3/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:25.233551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8594db-437b-409e-8c2c-4f4ad5a539ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc22323158714269b0da2b01121fe3ed, user_id=59ebb8f1e98d40c9aea425fad464f1e3, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:25.237635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8594db-437b-409e-8c2c-4f4ad5a539ed admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:25.472069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1109/2218] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:47:25 2022] PUT /identity/v3/projects/dc22323158714269b0da2b01121fe3ed/users/59ebb8f1e98d40c9aea425fad464f1e3/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:47:25.477151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61974130-f77d-45c6-9023-4e22bee884ea None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:25.477469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61974130-f77d-45c6-9023-4e22bee884ea None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:25.477748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61974130-f77d-45c6-9023-4e22bee884ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:25.499140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-61974130-f77d-45c6-9023-4e22bee884ea None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:25.501002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-61974130-f77d-45c6-9023-4e22bee884ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:25.542743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:47:25.566097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1110/2219] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:25.575698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8594db-437b-409e-8c2c-4f4ad5a539ed admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:25.577758 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7c8594db-437b-409e-8c2c-4f4ad5a539ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:25.645653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8594db-437b-409e-8c2c-4f4ad5a539ed admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:25.649827 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e3a7f-c11e-440e-b551-7441159e0b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:25.650667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e3a7f-c11e-440e-b551-7441159e0b75 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:25.651056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e3a7f-c11e-440e-b551-7441159e0b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:25.651438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e3a7f-c11e-440e-b551-7441159e0b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:25.662004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e3a7f-c11e-440e-b551-7441159e0b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:25.663392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e3a7f-c11e-440e-b551-7441159e0b75 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:25.677030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1110/2220] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:47:29.843679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ef463-2c4d-4a34-bdce-984d1c784d84 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:29.844244 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ef463-2c4d-4a34-bdce-984d1c784d84 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:29.844346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ef463-2c4d-4a34-bdce-984d1c784d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:29.863016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-bd2ef463-2c4d-4a34-bdce-984d1c784d84 None None] MFA Rules not processed for user `8773c88f88614a9d903f667305a217e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:29.864303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-bd2ef463-2c4d-4a34-bdce-984d1c784d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:29.920994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1111/2221] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:47:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:47:32.340094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b02c0-12d1-49f5-a220-c8d873d0e09d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:32.340594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b02c0-12d1-49f5-a220-c8d873d0e09d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:32.340684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b02c0-12d1-49f5-a220-c8d873d0e09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:32.372722 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-364b02c0-12d1-49f5-a220-c8d873d0e09d None None] MFA Rules not processed for user `59ebb8f1e98d40c9aea425fad464f1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:32.374000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-364b02c0-12d1-49f5-a220-c8d873d0e09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:32.474093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1111/2222] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:47:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:32.478723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ef463-2c4d-4a34-bdce-984d1c784d84 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:32.489286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ef463-2c4d-4a34-bdce-984d1c784d84 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:32.496874 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9a01e-46f8-4282-9475-3dbaa1abaa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:32.497829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9a01e-46f8-4282-9475-3dbaa1abaa95 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:32.508765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9a01e-46f8-4282-9475-3dbaa1abaa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:32.508765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9a01e-46f8-4282-9475-3dbaa1abaa95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:32.508765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9a01e-46f8-4282-9475-3dbaa1abaa95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:32.508765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9a01e-46f8-4282-9475-3dbaa1abaa95 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:32.587040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1112/2223] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:47:32 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:47:32.591003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b02c0-12d1-49f5-a220-c8d873d0e09d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:32.600256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:47:32.603354 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b02c0-12d1-49f5-a220-c8d873d0e09d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:32.666924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b32341-2373-47c2-aacc-300c7084f7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:32.667609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b32341-2373-47c2-aacc-300c7084f7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:32.667898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b32341-2373-47c2-aacc-300c7084f7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:32.668178 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b32341-2373-47c2-aacc-300c7084f7b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:32.669045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b32341-2373-47c2-aacc-300c7084f7b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:32.671075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b32341-2373-47c2-aacc-300c7084f7b7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:32.768970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1112/2224] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:47:32 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:47:32.773082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9a01e-46f8-4282-9475-3dbaa1abaa95 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:32.784318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9a01e-46f8-4282-9475-3dbaa1abaa95 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:32.789219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da28d2-7abd-4cdb-95bb-7e1ea6dfd1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:32.795070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da28d2-7abd-4cdb-95bb-7e1ea6dfd1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:32.795070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da28d2-7abd-4cdb-95bb-7e1ea6dfd1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:32.795070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da28d2-7abd-4cdb-95bb-7e1ea6dfd1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:32.795070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da28d2-7abd-4cdb-95bb-7e1ea6dfd1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:32.795070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da28d2-7abd-4cdb-95bb-7e1ea6dfd1f4 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:32.804853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1113/2225] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:47:32.809096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b32341-2373-47c2-aacc-300c7084f7b7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:32.822999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b32341-2373-47c2-aacc-300c7084f7b7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:32.832159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303dfb-b7e6-4fb0-bbd0-099d5cec3197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:32.833285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303dfb-b7e6-4fb0-bbd0-099d5cec3197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:32.833640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303dfb-b7e6-4fb0-bbd0-099d5cec3197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:32.833978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303dfb-b7e6-4fb0-bbd0-099d5cec3197 admin admin] PATH_INFO: `/v3/projects/3234cee85a5f42018aecf82fff4ad600/users/a8b752c6c8f7421786bfa2e8e52619d6/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:32.851463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b303dfb-b7e6-4fb0-bbd0-099d5cec3197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3234cee85a5f42018aecf82fff4ad600, user_id=a8b752c6c8f7421786bfa2e8e52619d6, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:32.854210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b303dfb-b7e6-4fb0-bbd0-099d5cec3197 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:32.950905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1113/2226] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:47:32 2022] PUT /identity/v3/projects/3234cee85a5f42018aecf82fff4ad600/users/a8b752c6c8f7421786bfa2e8e52619d6/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:47:32.954956 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da28d2-7abd-4cdb-95bb-7e1ea6dfd1f4 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:32.967550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da28d2-7abd-4cdb-95bb-7e1ea6dfd1f4 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:33.036939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0f256-f4ed-4be2-b90a-dd8d6ef46802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:33.037790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0f256-f4ed-4be2-b90a-dd8d6ef46802 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:33.038079 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0f256-f4ed-4be2-b90a-dd8d6ef46802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:33.038367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0f256-f4ed-4be2-b90a-dd8d6ef46802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:33.039098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0f256-f4ed-4be2-b90a-dd8d6ef46802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:33.040710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0f256-f4ed-4be2-b90a-dd8d6ef46802 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:33.056586 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1114/2227] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:47:33.058127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303dfb-b7e6-4fb0-bbd0-099d5cec3197 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:33.070534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303dfb-b7e6-4fb0-bbd0-099d5cec3197 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:33.082210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c24fa0-4847-4fc6-9aaf-79d8237ed653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:33.083125 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c24fa0-4847-4fc6-9aaf-79d8237ed653 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:33.083410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c24fa0-4847-4fc6-9aaf-79d8237ed653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:33.083687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c24fa0-4847-4fc6-9aaf-79d8237ed653 admin admin] PATH_INFO: `/v3/projects/3234cee85a5f42018aecf82fff4ad600/users/a8b752c6c8f7421786bfa2e8e52619d6/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:33.085770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c24fa0-4847-4fc6-9aaf-79d8237ed653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3234cee85a5f42018aecf82fff4ad600, user_id=a8b752c6c8f7421786bfa2e8e52619d6, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:33.088500 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c24fa0-4847-4fc6-9aaf-79d8237ed653 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:33.101055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1114/2228] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:47:33 2022] PUT /identity/v3/projects/3234cee85a5f42018aecf82fff4ad600/users/a8b752c6c8f7421786bfa2e8e52619d6/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:47:41.209399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc3a4-c12b-4413-95fb-841ede55a6e1 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:41.209908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc3a4-c12b-4413-95fb-841ede55a6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:41.210007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc3a4-c12b-4413-95fb-841ede55a6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:41.242075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-508dc3a4-c12b-4413-95fb-841ede55a6e1 None None] MFA Rules not processed for user `a8b752c6c8f7421786bfa2e8e52619d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:41.243862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-508dc3a4-c12b-4413-95fb-841ede55a6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:41.309818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:47:41.346208 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1115/2229] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:47:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:41.358083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c24fa0-4847-4fc6-9aaf-79d8237ed653 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:41.359830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-40c24fa0-4847-4fc6-9aaf-79d8237ed653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:41.422937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c24fa0-4847-4fc6-9aaf-79d8237ed653 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:41.427172 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029febb-0282-4fd7-8599-8180560854c4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1a7213cbf6448b7ba6570a52a4c7c0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:41.427821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029febb-0282-4fd7-8599-8180560854c4 service glance] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:41.428233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029febb-0282-4fd7-8599-8180560854c4 service glance] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:41.428429 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029febb-0282-4fd7-8599-8180560854c4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:41.486557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029febb-0282-4fd7-8599-8180560854c4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:41.488045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029febb-0282-4fd7-8599-8180560854c4 service glance] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:41.496823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1115/2230] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:47:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:47:41.781042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d45592-b169-497e-a2e4-0730832ab184 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:41.781432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d45592-b169-497e-a2e4-0730832ab184 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:41.781812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d45592-b169-497e-a2e4-0730832ab184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:41.799956 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e6d45592-b169-497e-a2e4-0730832ab184 None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:41.801324 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e6d45592-b169-497e-a2e4-0730832ab184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:41.863834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1116/2231] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:47:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:41.877237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029febb-0282-4fd7-8599-8180560854c4 service glance] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:41.878687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f029febb-0282-4fd7-8599-8180560854c4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:41.892954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:47:41.966746 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029febb-0282-4fd7-8599-8180560854c4 service glance] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:41.972175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058ad14-1903-47dc-accd-cf50dabd28f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:41.972828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058ad14-1903-47dc-accd-cf50dabd28f5 service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:41.978503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058ad14-1903-47dc-accd-cf50dabd28f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:41.978503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058ad14-1903-47dc-accd-cf50dabd28f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:41.982296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058ad14-1903-47dc-accd-cf50dabd28f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:41.983979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058ad14-1903-47dc-accd-cf50dabd28f5 service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:41.996370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1116/2232] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:47:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:47:51.856498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=36 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:47:52.071512 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=27 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:47:52.879622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008573c4-2503-4d89-95a4-5a89f22300e1 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:52.880124 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008573c4-2503-4d89-95a4-5a89f22300e1 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:52.880245 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008573c4-2503-4d89-95a4-5a89f22300e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:52.899519 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-008573c4-2503-4d89-95a4-5a89f22300e1 None None] MFA Rules not processed for user `59ebb8f1e98d40c9aea425fad464f1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:52.900718 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-008573c4-2503-4d89-95a4-5a89f22300e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:52.907786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1117/2233] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:47:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:47:56.802621 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338222e8-afe7-460f-9db9-b71a8c921b4e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:56.803145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338222e8-afe7-460f-9db9-b71a8c921b4e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:56.803145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338222e8-afe7-460f-9db9-b71a8c921b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:56.823649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-338222e8-afe7-460f-9db9-b71a8c921b4e None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:47:56.824927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-338222e8-afe7-460f-9db9-b71a8c921b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:47:56.832880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1117/2234] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:47:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:47:56.850069 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008573c4-2503-4d89-95a4-5a89f22300e1 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:47:56.863977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008573c4-2503-4d89-95a4-5a89f22300e1 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:47:56.871008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81675-e35a-4259-bcb8-782077294e25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:47:56.871770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81675-e35a-4259-bcb8-782077294e25 service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:47:56.872099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81675-e35a-4259-bcb8-782077294e25 service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:47:56.872460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81675-e35a-4259-bcb8-782077294e25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:47:56.882541 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd81675-e35a-4259-bcb8-782077294e25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:47:56.884269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd81675-e35a-4259-bcb8-782077294e25 service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:47:56.900736 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1118/2235] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:47:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:48:06.817467 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:48:06.892671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=9 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:48:17.447929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c941c6-d988-4551-a1f9-e77d9fdd05b6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:48:17.448413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c941c6-d988-4551-a1f9-e77d9fdd05b6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:48:17.448518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c941c6-d988-4551-a1f9-e77d9fdd05b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:48:17.468001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-94c941c6-d988-4551-a1f9-e77d9fdd05b6 None None] MFA Rules not processed for user `899cbac79d874b308b500651bb40b2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:48:17.469271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-94c941c6-d988-4551-a1f9-e77d9fdd05b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:48:17.526543 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1118/2236] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:48:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:48:27.029341 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81675-e35a-4259-bcb8-782077294e25 service swift] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:48:27.094261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81675-e35a-4259-bcb8-782077294e25 service swift] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:48:27.098929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e921f9-fb9b-4e22-8dc9-300766f062c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:48:27.099804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e921f9-fb9b-4e22-8dc9-300766f062c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:48:27.100095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e921f9-fb9b-4e22-8dc9-300766f062c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:48:27.100542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e921f9-fb9b-4e22-8dc9-300766f062c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:48:27.110721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e921f9-fb9b-4e22-8dc9-300766f062c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:48:27.112234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e921f9-fb9b-4e22-8dc9-300766f062c8 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:48:27.122536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1119/2237] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:48:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:48:27.519385 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=25 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:48:37.117937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=26 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:48:37.898937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf775c9-7df6-4427-b1c4-c48cc8736011 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:48:37.899499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf775c9-7df6-4427-b1c4-c48cc8736011 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:48:37.899499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf775c9-7df6-4427-b1c4-c48cc8736011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:48:37.917988 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-bbf775c9-7df6-4427-b1c4-c48cc8736011 None None] MFA Rules not processed for user `246f07024efa444da74f2b898f426b46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:48:37.919224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bbf775c9-7df6-4427-b1c4-c48cc8736011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:48:37.973744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1119/2238] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:48:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:48:47.966992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=25 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:48:54.027590 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d858c8-1f42-4b6f-8819-9e06bb595bae None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:48:54.028266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d858c8-1f42-4b6f-8819-9e06bb595bae None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:48:54.028266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d858c8-1f42-4b6f-8819-9e06bb595bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:48:54.046334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-f4d858c8-1f42-4b6f-8819-9e06bb595bae None None] MFA Rules not processed for user `8773c88f88614a9d903f667305a217e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:48:54.047552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f4d858c8-1f42-4b6f-8819-9e06bb595bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:48:54.102550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1120/2239] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:48:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:48:55.763816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813a18f-1fb8-4ff2-af45-4cac44c4090f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:48:55.764346 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813a18f-1fb8-4ff2-af45-4cac44c4090f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:48:55.764453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813a18f-1fb8-4ff2-af45-4cac44c4090f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:48:55.782094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5813a18f-1fb8-4ff2-af45-4cac44c4090f None None] MFA Rules not processed for user `7bf3f705e04640a2a5d9bedb71c6ccb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:48:55.783279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5813a18f-1fb8-4ff2-af45-4cac44c4090f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:48:55.841641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1120/2240] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:48:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:49:04.095737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=25 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:49:05.834366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=25 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:49:11.208361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d858c8-1f42-4b6f-8819-9e06bb595bae None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:11.209860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f4d858c8-1f42-4b6f-8819-9e06bb595bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:11.223558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d858c8-1f42-4b6f-8819-9e06bb595bae None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:11.227134 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77604fde-e9f1-4544-9ae0-d32d586534a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:11.227824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77604fde-e9f1-4544-9ae0-d32d586534a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:11.228108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77604fde-e9f1-4544-9ae0-d32d586534a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:11.228459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77604fde-e9f1-4544-9ae0-d32d586534a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:11.230796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-77604fde-e9f1-4544-9ae0-d32d586534a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:11.313580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77604fde-e9f1-4544-9ae0-d32d586534a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:11.315036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77604fde-e9f1-4544-9ae0-d32d586534a2 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:11.326464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1121/2241] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:49:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:49:16.172140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813a18f-1fb8-4ff2-af45-4cac44c4090f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:16.181810 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813a18f-1fb8-4ff2-af45-4cac44c4090f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:16.185690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee204964-6365-4fa8-a126-4da6e2fa89db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:16.186337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee204964-6365-4fa8-a126-4da6e2fa89db service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:16.186626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee204964-6365-4fa8-a126-4da6e2fa89db service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:16.186906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee204964-6365-4fa8-a126-4da6e2fa89db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:16.188729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ee204964-6365-4fa8-a126-4da6e2fa89db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:16.202569 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee204964-6365-4fa8-a126-4da6e2fa89db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:16.204307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee204964-6365-4fa8-a126-4da6e2fa89db service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:16.217097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1121/2242] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:49:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:49:17.468602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77604fde-e9f1-4544-9ae0-d32d586534a2 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:17.472158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-77604fde-e9f1-4544-9ae0-d32d586534a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:17.485130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77604fde-e9f1-4544-9ae0-d32d586534a2 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:17.490841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b89365-1fc7-43e5-99bb-49ff1507e0c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:17.491560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b89365-1fc7-43e5-99bb-49ff1507e0c3 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:17.491906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b89365-1fc7-43e5-99bb-49ff1507e0c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:17.492246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b89365-1fc7-43e5-99bb-49ff1507e0c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:17.494209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-03b89365-1fc7-43e5-99bb-49ff1507e0c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:17.504884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b89365-1fc7-43e5-99bb-49ff1507e0c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:17.506492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b89365-1fc7-43e5-99bb-49ff1507e0c3 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:17.516358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1122/2243] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:49:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:49:19.926233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee204964-6365-4fa8-a126-4da6e2fa89db service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:19.934791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee204964-6365-4fa8-a126-4da6e2fa89db service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:19.938528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61537458-5a18-47a9-b7f2-0a200cd11ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:19.939170 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537458-5a18-47a9-b7f2-0a200cd11ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:19.939446 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537458-5a18-47a9-b7f2-0a200cd11ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:19.939716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537458-5a18-47a9-b7f2-0a200cd11ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:19.941596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-61537458-5a18-47a9-b7f2-0a200cd11ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:19.951211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61537458-5a18-47a9-b7f2-0a200cd11ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:19.952618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61537458-5a18-47a9-b7f2-0a200cd11ce7 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:19.960490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1122/2244] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:49:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:49:20.424118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b89365-1fc7-43e5-99bb-49ff1507e0c3 service nova] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:20.425583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-03b89365-1fc7-43e5-99bb-49ff1507e0c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:20.496192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b89365-1fc7-43e5-99bb-49ff1507e0c3 service nova] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:20.501177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dfc60-7ab3-4e5d-a245-944c7fa6f657 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:20.501820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dfc60-7ab3-4e5d-a245-944c7fa6f657 service placement] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:20.502099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dfc60-7ab3-4e5d-a245-944c7fa6f657 service placement] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:20.502369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dfc60-7ab3-4e5d-a245-944c7fa6f657 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:20.504143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a55dfc60-7ab3-4e5d-a245-944c7fa6f657 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:20.516316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55dfc60-7ab3-4e5d-a245-944c7fa6f657 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:20.517924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55dfc60-7ab3-4e5d-a245-944c7fa6f657 service placement] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:20.528730 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1123/2245] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:49:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:49:29.957041 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=12 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:49:30.523892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=72 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:49:39.743987 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61537458-5a18-47a9-b7f2-0a200cd11ce7 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:39.745355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-61537458-5a18-47a9-b7f2-0a200cd11ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:39.761138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61537458-5a18-47a9-b7f2-0a200cd11ce7 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:39.767232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0762731-9fa9-477e-90ab-b74eda0beb2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:39.767876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0762731-9fa9-477e-90ab-b74eda0beb2f service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:39.768185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0762731-9fa9-477e-90ab-b74eda0beb2f service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:39.768457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0762731-9fa9-477e-90ab-b74eda0beb2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:39.770199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f0762731-9fa9-477e-90ab-b74eda0beb2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:39.780468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0762731-9fa9-477e-90ab-b74eda0beb2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:39.782138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0762731-9fa9-477e-90ab-b74eda0beb2f service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:39.790823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1123/2246] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:49:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:49:45.332484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dfc60-7ab3-4e5d-a245-944c7fa6f657 service placement] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:45.346357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dfc60-7ab3-4e5d-a245-944c7fa6f657 service placement] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:45.351890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65caee5-64ff-49c6-adb2-f5f260a82162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:45.352548 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65caee5-64ff-49c6-adb2-f5f260a82162 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:45.352828 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65caee5-64ff-49c6-adb2-f5f260a82162 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:45.353102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65caee5-64ff-49c6-adb2-f5f260a82162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:45.354952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d65caee5-64ff-49c6-adb2-f5f260a82162 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:45.369562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65caee5-64ff-49c6-adb2-f5f260a82162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:45.371379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65caee5-64ff-49c6-adb2-f5f260a82162 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:45.383235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1124/2247] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:49:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:49:46.178357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16e90f-c7aa-4a2a-b958-95652ea9ef35 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:46.178840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16e90f-c7aa-4a2a-b958-95652ea9ef35 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:46.178945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16e90f-c7aa-4a2a-b958-95652ea9ef35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:46.196700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4b16e90f-c7aa-4a2a-b958-95652ea9ef35 None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:49:46.197958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4b16e90f-c7aa-4a2a-b958-95652ea9ef35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:46.206145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1124/2248] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:49:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:49:46.247151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65caee5-64ff-49c6-adb2-f5f260a82162 service nova] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:46.258507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65caee5-64ff-49c6-adb2-f5f260a82162 service nova] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:46.267597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b62a-474d-4916-8bdf-6337735ee3a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:46.267597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b62a-474d-4916-8bdf-6337735ee3a8 service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:46.267897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b62a-474d-4916-8bdf-6337735ee3a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:46.268128 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b62a-474d-4916-8bdf-6337735ee3a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:46.277856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968b62a-474d-4916-8bdf-6337735ee3a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:46.279445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968b62a-474d-4916-8bdf-6337735ee3a8 service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:46.287598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1125/2249] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:49:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:49:56.191405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=9 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:49:56.285115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=12 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:49:58.070919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16e90f-c7aa-4a2a-b958-95652ea9ef35 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:58.081502 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16e90f-c7aa-4a2a-b958-95652ea9ef35 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:58.087510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe299b-281f-45b2-a23e-d55df57fe780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:58.087510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe299b-281f-45b2-a23e-d55df57fe780 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:58.087510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe299b-281f-45b2-a23e-d55df57fe780 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:58.087510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe299b-281f-45b2-a23e-d55df57fe780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:58.090174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2cfe299b-281f-45b2-a23e-d55df57fe780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:58.101286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe299b-281f-45b2-a23e-d55df57fe780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:58.103026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe299b-281f-45b2-a23e-d55df57fe780 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:58.113306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1125/2250] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:49:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:49:58.295900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b62a-474d-4916-8bdf-6337735ee3a8 service swift] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:58.314194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b62a-474d-4916-8bdf-6337735ee3a8 service swift] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:58.326383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4b730-b920-4216-8fe5-ca74db26fc6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:58.327301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4b730-b920-4216-8fe5-ca74db26fc6e service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:58.327798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4b730-b920-4216-8fe5-ca74db26fc6e service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:58.328358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4b730-b920-4216-8fe5-ca74db26fc6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:58.331149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-19d4b730-b920-4216-8fe5-ca74db26fc6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:58.350131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4b730-b920-4216-8fe5-ca74db26fc6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:58.352816 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4b730-b920-4216-8fe5-ca74db26fc6e service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:58.401626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1126/2251] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:49:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:49:59.745416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe299b-281f-45b2-a23e-d55df57fe780 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:49:59.758127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe299b-281f-45b2-a23e-d55df57fe780 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:49:59.764218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5119b-d33b-467b-b587-6572ef17a817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:49:59.764846 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5119b-d33b-467b-b587-6572ef17a817 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:49:59.765123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5119b-d33b-467b-b587-6572ef17a817 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:49:59.765433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5119b-d33b-467b-b587-6572ef17a817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:49:59.767204 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3bf5119b-d33b-467b-b587-6572ef17a817 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:49:59.778559 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5119b-d33b-467b-b587-6572ef17a817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:49:59.780117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5119b-d33b-467b-b587-6572ef17a817 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:49:59.789288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1126/2252] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:49:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:04.620552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be13a-f57f-4443-8a38-2f8fc582d495 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:04.625033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be13a-f57f-4443-8a38-2f8fc582d495 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:04.625033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be13a-f57f-4443-8a38-2f8fc582d495 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:04.625033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1127/2253] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:50:04.628936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d131e-4262-4cda-ab28-9fbfcee9e5a2 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:04.629734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d131e-4262-4cda-ab28-9fbfcee9e5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:04.630071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d131e-4262-4cda-ab28-9fbfcee9e5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:04.652109 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5a8d131e-4262-4cda-ab28-9fbfcee9e5a2 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:04.655907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5a8d131e-4262-4cda-ab28-9fbfcee9e5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:04.666742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1127/2254] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:50:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:50:04.679334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be13a-f57f-4443-8a38-2f8fc582d495 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:04.697394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be13a-f57f-4443-8a38-2f8fc582d495 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:04.701187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d619-a2a5-4cd6-8167-5122237e0867 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1a7213cbf6448b7ba6570a52a4c7c0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:04.701853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d619-a2a5-4cd6-8167-5122237e0867 service glance] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:04.702130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d619-a2a5-4cd6-8167-5122237e0867 service glance] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:04.702402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d619-a2a5-4cd6-8167-5122237e0867 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:04.713047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3d619-a2a5-4cd6-8167-5122237e0867 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:04.714507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3d619-a2a5-4cd6-8167-5122237e0867 service glance] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:04.725914 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1128/2255] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:04.957220 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12c7a-ac3e-4c4e-a8f0-748e2547fec6 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:04.957457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12c7a-ac3e-4c4e-a8f0-748e2547fec6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:04.957709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12c7a-ac3e-4c4e-a8f0-748e2547fec6 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:04.959114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1128/2256] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:50:04.963976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153425-95a0-48d9-9bf4-98d1362c8597 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:04.964266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153425-95a0-48d9-9bf4-98d1362c8597 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:04.964572 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153425-95a0-48d9-9bf4-98d1362c8597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:04.983354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-29153425-95a0-48d9-9bf4-98d1362c8597 None None] MFA Rules not processed for user `481ef56b1cfa465a8efc59736a34e412`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:04.984567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-29153425-95a0-48d9-9bf4-98d1362c8597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:04.991521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1129/2257] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:50:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:50:05.003398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb12c7a-ac3e-4c4e-a8f0-748e2547fec6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:05.015583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb12c7a-ac3e-4c4e-a8f0-748e2547fec6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:05.019274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa8dad-d754-43ec-a4f4-b75597e7f872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:05.019910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa8dad-d754-43ec-a4f4-b75597e7f872 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:05.020191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa8dad-d754-43ec-a4f4-b75597e7f872 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:05.020466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa8dad-d754-43ec-a4f4-b75597e7f872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:05.028763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa8dad-d754-43ec-a4f4-b75597e7f872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:05.030274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa8dad-d754-43ec-a4f4-b75597e7f872 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:05.038515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1129/2258] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:05.121546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5b67-f7c6-44f9-9c2e-0a1ef77c6f29 None None] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:05.122135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5b67-f7c6-44f9-9c2e-0a1ef77c6f29 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:05.122629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5b67-f7c6-44f9-9c2e-0a1ef77c6f29 None None] PATH_INFO: `/` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:05.124071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1130/2259] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:50:05.129930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10caa05-4be5-4fef-858b-ffd4e14801e1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:05.130332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10caa05-4be5-4fef-858b-ffd4e14801e1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:05.130695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10caa05-4be5-4fef-858b-ffd4e14801e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:05.150857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d10caa05-4be5-4fef-858b-ffd4e14801e1 None None] MFA Rules not processed for user `147d764ef17042dbb7ea179c29711588`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:05.152175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d10caa05-4be5-4fef-858b-ffd4e14801e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:05.163887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1130/2260] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:50:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:05.175853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd5b67-f7c6-44f9-9c2e-0a1ef77c6f29 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:05.187269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd5b67-f7c6-44f9-9c2e-0a1ef77c6f29 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:05.190997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1765ed-ba5a-451d-90d9-b5a6f860d860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:05.191639 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1765ed-ba5a-451d-90d9-b5a6f860d860 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:05.191927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1765ed-ba5a-451d-90d9-b5a6f860d860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:05.192205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1765ed-ba5a-451d-90d9-b5a6f860d860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:05.203473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1765ed-ba5a-451d-90d9-b5a6f860d860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:05.204906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1765ed-ba5a-451d-90d9-b5a6f860d860 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:05.216799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1131/2261] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:09.870441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834d8ec-0765-4e74-8134-90cadcffb930 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:09.870951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834d8ec-0765-4e74-8134-90cadcffb930 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:09.871039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834d8ec-0765-4e74-8134-90cadcffb930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:09.888744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0834d8ec-0765-4e74-8134-90cadcffb930 None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:09.893257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0834d8ec-0765-4e74-8134-90cadcffb930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:09.897444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1131/2262] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:50:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:09.910009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1765ed-ba5a-451d-90d9-b5a6f860d860 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:09.922078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1765ed-ba5a-451d-90d9-b5a6f860d860 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:09.927160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633387a-97d0-4a37-8562-de0f921a446c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:09.927800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633387a-97d0-4a37-8562-de0f921a446c service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:09.928089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633387a-97d0-4a37-8562-de0f921a446c service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:09.928366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633387a-97d0-4a37-8562-de0f921a446c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:09.936623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633387a-97d0-4a37-8562-de0f921a446c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:09.938240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633387a-97d0-4a37-8562-de0f921a446c service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:09.946405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1132/2263] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:50:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:50:19.883319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=27 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:19.943058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=27 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:20.577091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834d8ec-0765-4e74-8134-90cadcffb930 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:20.591005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834d8ec-0765-4e74-8134-90cadcffb930 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:20.598720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1944d-d236-4dc5-a093-a3a2e7c65dce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:20.599652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1944d-d236-4dc5-a093-a3a2e7c65dce service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:20.599936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1944d-d236-4dc5-a093-a3a2e7c65dce service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:20.600236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1944d-d236-4dc5-a093-a3a2e7c65dce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:20.602062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b8e1944d-d236-4dc5-a093-a3a2e7c65dce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:20.621034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1944d-d236-4dc5-a093-a3a2e7c65dce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:20.622786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1944d-d236-4dc5-a093-a3a2e7c65dce service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:20.632365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1132/2264] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:25.703290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e09d3-ed07-4ac3-b9cf-fba37db12b2e None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:25.704791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e09d3-ed07-4ac3-b9cf-fba37db12b2e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:25.705595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e09d3-ed07-4ac3-b9cf-fba37db12b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:25.725377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-cd8e09d3-ed07-4ac3-b9cf-fba37db12b2e None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:25.726589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cd8e09d3-ed07-4ac3-b9cf-fba37db12b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:25.734240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1133/2265] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jul 14 11:50:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:25.747575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1944d-d236-4dc5-a093-a3a2e7c65dce service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:25.759497 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1944d-d236-4dc5-a093-a3a2e7c65dce service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:25.764522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a294a47-06ca-4196-ac6c-df08ad7c2c6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:25.765202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a294a47-06ca-4196-ac6c-df08ad7c2c6e service swift] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:25.765480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a294a47-06ca-4196-ac6c-df08ad7c2c6e service swift] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:25.765760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a294a47-06ca-4196-ac6c-df08ad7c2c6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:25.774161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a294a47-06ca-4196-ac6c-df08ad7c2c6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:25.775747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a294a47-06ca-4196-ac6c-df08ad7c2c6e service swift] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:25.783955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1133/2266] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:50:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:50:26.751481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e09d3-ed07-4ac3-b9cf-fba37db12b2e None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:26.761589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e09d3-ed07-4ac3-b9cf-fba37db12b2e None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:26.765482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40f2ea-1c63-4b3a-b708-2a3106b65cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:26.766211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40f2ea-1c63-4b3a-b708-2a3106b65cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:26.766653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40f2ea-1c63-4b3a-b708-2a3106b65cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:26.767003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40f2ea-1c63-4b3a-b708-2a3106b65cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:26.768934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2b40f2ea-1c63-4b3a-b708-2a3106b65cd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:26.848302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40f2ea-1c63-4b3a-b708-2a3106b65cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:26.850463 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40f2ea-1c63-4b3a-b708-2a3106b65cd9 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:26.861865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1134/2267] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:31.225592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3005896-04e5-488d-b1a0-82f25efd0b83 None None] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:31.226078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3005896-04e5-488d-b1a0-82f25efd0b83 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:31.226279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3005896-04e5-488d-b1a0-82f25efd0b83 None None] PATH_INFO: `/` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:31.227760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1134/2268] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jul 14 11:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:50:31.238251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0200ba-d62f-4630-92f8-c0d5b446d5d8 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:31.238714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0200ba-d62f-4630-92f8-c0d5b446d5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:31.238853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0200ba-d62f-4630-92f8-c0d5b446d5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:31.257330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-1b0200ba-d62f-4630-92f8-c0d5b446d5d8 None None] MFA Rules not processed for user `fa879d581c6d44e18a8703e66ffbd549`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:31.258555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1b0200ba-d62f-4630-92f8-c0d5b446d5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:31.267695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1135/2269] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jul 14 11:50:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:31.281322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3005896-04e5-488d-b1a0-82f25efd0b83 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:31.283098 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c3005896-04e5-488d-b1a0-82f25efd0b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:31.300859 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3005896-04e5-488d-b1a0-82f25efd0b83 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:31.305936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9565b8b-e863-45a6-a29c-9fc838437d77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:31.306585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9565b8b-e863-45a6-a29c-9fc838437d77 service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:31.306868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9565b8b-e863-45a6-a29c-9fc838437d77 service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:31.307146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9565b8b-e863-45a6-a29c-9fc838437d77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:31.316817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9565b8b-e863-45a6-a29c-9fc838437d77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:31.318809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9565b8b-e863-45a6-a29c-9fc838437d77 service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:31.328940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1135/2270] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:31.390142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0200ba-d62f-4630-92f8-c0d5b446d5d8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:31.392019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1b0200ba-d62f-4630-92f8-c0d5b446d5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:31.408578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0200ba-d62f-4630-92f8-c0d5b446d5d8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:31.413529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:31.414451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:31.414754 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:31.415028 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] PATH_INFO: `/v3/users/7bf3f705e04640a2a5d9bedb71c6ccb5` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:31.416223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf3f705e04640a2a5d9bedb71c6ccb5)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:31.418270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:31.640304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf3f705e04640a2a5d9bedb71c6ccb5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:31.640875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf3f705e04640a2a5d9bedb71c6ccb5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:31.654006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf3f705e04640a2a5d9bedb71c6ccb5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:31.662089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf3f705e04640a2a5d9bedb71c6ccb5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:31.741274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fa9fd90a-6f05-4dbb-a58d-0969f47fecb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf3f705e04640a2a5d9bedb71c6ccb5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:31.747899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1136/2271] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:50:31 2022] DELETE /identity/v3/users/7bf3f705e04640a2a5d9bedb71c6ccb5 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:32.008296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9565b8b-e863-45a6-a29c-9fc838437d77 service placement] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:32.036381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9565b8b-e863-45a6-a29c-9fc838437d77 service placement] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:32.094218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:32.095621 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:32.095939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:32.096216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] PATH_INFO: `/v3/projects/951609a482e34e93908057a68c5dc706` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:32.097477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=951609a482e34e93908057a68c5dc706)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:32.099382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:32.201813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '951609a482e34e93908057a68c5dc706', 'request_id': 'req-ec065147-c591-4844-9d98-d2dc98a5ba90'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:32.263654 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '951609a482e34e93908057a68c5dc706', 'request_id': 'req-ec065147-c591-4844-9d98-d2dc98a5ba90'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:32.270150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '951609a482e34e93908057a68c5dc706', 'request_id': 'req-ec065147-c591-4844-9d98-d2dc98a5ba90'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:32.282737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] The token cache is being invalidate because project 951609a482e34e93908057a68c5dc706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:50:32.283005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:32.283655 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ec065147-c591-4844-9d98-d2dc98a5ba90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:32.285247 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1136/2272] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:50:32 2022] DELETE /identity/v3/projects/951609a482e34e93908057a68c5dc706 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:32.290178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177291df-b752-4ff4-b322-cb14c1a5da99 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:32.290636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177291df-b752-4ff4-b322-cb14c1a5da99 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:32.290772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177291df-b752-4ff4-b322-cb14c1a5da99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:32.309025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-177291df-b752-4ff4-b322-cb14c1a5da99 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:32.310252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-177291df-b752-4ff4-b322-cb14c1a5da99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:32.355337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:32.367363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1137/2273] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:32.376624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e9e92-848b-438d-924c-68537949568c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:32.376911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e9e92-848b-438d-924c-68537949568c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:32.377215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e9e92-848b-438d-924c-68537949568c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:32.396126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c00e9e92-848b-438d-924c-68537949568c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:32.398526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c00e9e92-848b-438d-924c-68537949568c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:32.406158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1137/2274] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:32.411127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177291df-b752-4ff4-b322-cb14c1a5da99 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:32.421560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177291df-b752-4ff4-b322-cb14c1a5da99 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:32.426649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3ee6e-0a55-4240-96dd-0438a7b2a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:32.427448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3ee6e-0a55-4240-96dd-0438a7b2a221 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:32.427576 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3ee6e-0a55-4240-96dd-0438a7b2a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:32.427851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3ee6e-0a55-4240-96dd-0438a7b2a221 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:32.428580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3ee6e-0a55-4240-96dd-0438a7b2a221 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:50:32.428889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3ee6e-0a55-4240-96dd-0438a7b2a221 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:32.430536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3ee6e-0a55-4240-96dd-0438a7b2a221 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:32.442059 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1138/2275] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:50:32.446645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e9e92-848b-438d-924c-68537949568c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:32.456563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e9e92-848b-438d-924c-68537949568c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:32.461199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea7341-263a-419a-898e-1d942a28ba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:32.461842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea7341-263a-419a-898e-1d942a28ba4f admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:32.462163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea7341-263a-419a-898e-1d942a28ba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:32.462440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea7341-263a-419a-898e-1d942a28ba4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:32.463133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea7341-263a-419a-898e-1d942a28ba4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:32.465039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea7341-263a-419a-898e-1d942a28ba4f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:32.529043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1138/2276] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:50:32 2022] POST /identity/v3/projects => generated 367 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:32.534050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3ee6e-0a55-4240-96dd-0438a7b2a221 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:32.544138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3ee6e-0a55-4240-96dd-0438a7b2a221 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:32.600799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a48077-ee8f-4ed0-8d86-99004b7af18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:32.601487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a48077-ee8f-4ed0-8d86-99004b7af18b admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:32.601770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a48077-ee8f-4ed0-8d86-99004b7af18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:32.602041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a48077-ee8f-4ed0-8d86-99004b7af18b admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:32.602720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a48077-ee8f-4ed0-8d86-99004b7af18b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:32.604639 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a48077-ee8f-4ed0-8d86-99004b7af18b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:32.658364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1139/2277] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:50:32 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:50:32.662467 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea7341-263a-419a-898e-1d942a28ba4f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:32.673542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea7341-263a-419a-898e-1d942a28ba4f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:32.678195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372e369-4bd8-400f-9182-f9daa919cfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:32.678873 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372e369-4bd8-400f-9182-f9daa919cfdd admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:32.679138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372e369-4bd8-400f-9182-f9daa919cfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:32.679408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372e369-4bd8-400f-9182-f9daa919cfdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:32.680165 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372e369-4bd8-400f-9182-f9daa919cfdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:32.681792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372e369-4bd8-400f-9182-f9daa919cfdd admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:32.692600 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1139/2278] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:32.696818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a48077-ee8f-4ed0-8d86-99004b7af18b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:32.707740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a48077-ee8f-4ed0-8d86-99004b7af18b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:32.712257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a92f2-aea1-4b9c-9462-7509b3ccb90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:32.712911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a92f2-aea1-4b9c-9462-7509b3ccb90d admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:32.713214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a92f2-aea1-4b9c-9462-7509b3ccb90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:32.713495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a92f2-aea1-4b9c-9462-7509b3ccb90d admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:32.714215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a92f2-aea1-4b9c-9462-7509b3ccb90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:32.715795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a92f2-aea1-4b9c-9462-7509b3ccb90d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:32.726592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1140/2279] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:32.730643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372e369-4bd8-400f-9182-f9daa919cfdd admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:32.741123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372e369-4bd8-400f-9182-f9daa919cfdd admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:32.745664 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a77ca-1dec-4d8c-9606-ba93f881851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:32.746557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a77ca-1dec-4d8c-9606-ba93f881851f admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:32.746831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a77ca-1dec-4d8c-9606-ba93f881851f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:32.747096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a77ca-1dec-4d8c-9606-ba93f881851f admin admin] PATH_INFO: `/v3/projects/0931133a75484c64b26b6c00146e29bd/users/c9990084b9214921bcf24b6d6b34a39c/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:32.755689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:32.762481 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a77ca-1dec-4d8c-9606-ba93f881851f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0931133a75484c64b26b6c00146e29bd, user_id=c9990084b9214921bcf24b6d6b34a39c, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:32.765089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a77ca-1dec-4d8c-9606-ba93f881851f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:32.852950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1140/2280] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:32 2022] PUT /identity/v3/projects/0931133a75484c64b26b6c00146e29bd/users/c9990084b9214921bcf24b6d6b34a39c/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:32.863831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828e46d-81b7-4c95-ae3a-67a78ba642e5 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:32.864122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828e46d-81b7-4c95-ae3a-67a78ba642e5 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:32.864408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828e46d-81b7-4c95-ae3a-67a78ba642e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:32.895011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c828e46d-81b7-4c95-ae3a-67a78ba642e5 None None] MFA Rules not processed for user `c9990084b9214921bcf24b6d6b34a39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:32.896260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c828e46d-81b7-4c95-ae3a-67a78ba642e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:32.970788 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1141/2281] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:32.981812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a77ca-1dec-4d8c-9606-ba93f881851f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:32.983554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-825a77ca-1dec-4d8c-9606-ba93f881851f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:33.054646 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a77ca-1dec-4d8c-9606-ba93f881851f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:33.065504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657aa43-efb2-4eb2-92e4-d08cb5715e28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:33.066104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657aa43-efb2-4eb2-92e4-d08cb5715e28 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:33.066746 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657aa43-efb2-4eb2-92e4-d08cb5715e28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:33.067128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657aa43-efb2-4eb2-92e4-d08cb5715e28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:33.077206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657aa43-efb2-4eb2-92e4-d08cb5715e28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:33.079351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657aa43-efb2-4eb2-92e4-d08cb5715e28 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:33.090423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1141/2282] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:50:34.161060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828e46d-81b7-4c95-ae3a-67a78ba642e5 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:34.162751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c828e46d-81b7-4c95-ae3a-67a78ba642e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:34.248102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:34.299352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828e46d-81b7-4c95-ae3a-67a78ba642e5 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:34.320600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb5f04-729d-4e67-8430-67617b3d0f31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:34.321690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb5f04-729d-4e67-8430-67617b3d0f31 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:34.322327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb5f04-729d-4e67-8430-67617b3d0f31 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:34.322801 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb5f04-729d-4e67-8430-67617b3d0f31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:34.331048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5cfb5f04-729d-4e67-8430-67617b3d0f31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:34.352579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb5f04-729d-4e67-8430-67617b3d0f31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:34.354526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb5f04-729d-4e67-8430-67617b3d0f31 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:34.375004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1142/2283] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:36.066199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bade53-aaaf-4515-9795-3d9d24004734 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:36.066692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bade53-aaaf-4515-9795-3d9d24004734 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:36.066816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bade53-aaaf-4515-9795-3d9d24004734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:36.085248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f3bade53-aaaf-4515-9795-3d9d24004734 None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:36.086826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f3bade53-aaaf-4515-9795-3d9d24004734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:36.147567 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1142/2284] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jul 14 11:50:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:36.159713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb5f04-729d-4e67-8430-67617b3d0f31 service nova] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:36.161210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5cfb5f04-729d-4e67-8430-67617b3d0f31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:36.237952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb5f04-729d-4e67-8430-67617b3d0f31 service nova] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:36.243238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9617cc4-0b7d-4a86-9aca-9f9defcb0aaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:36.243973 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9617cc4-0b7d-4a86-9aca-9f9defcb0aaf service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:36.244320 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9617cc4-0b7d-4a86-9aca-9f9defcb0aaf service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:36.244659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9617cc4-0b7d-4a86-9aca-9f9defcb0aaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:36.253972 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9617cc4-0b7d-4a86-9aca-9f9defcb0aaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:36.255552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9617cc4-0b7d-4a86-9aca-9f9defcb0aaf service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:36.264068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1143/2285] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:50:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:50:41.346478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bade53-aaaf-4515-9795-3d9d24004734 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:41.348499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f3bade53-aaaf-4515-9795-3d9d24004734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:41.406114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bade53-aaaf-4515-9795-3d9d24004734 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:41.410253 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85960212-2ce6-437a-a4bc-45bb7f499a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:41.410904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85960212-2ce6-437a-a4bc-45bb7f499a72 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:41.411190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85960212-2ce6-437a-a4bc-45bb7f499a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:41.411466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85960212-2ce6-437a-a4bc-45bb7f499a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:41.413299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-85960212-2ce6-437a-a4bc-45bb7f499a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:41.467528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:41.486232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85960212-2ce6-437a-a4bc-45bb7f499a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:41.488017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85960212-2ce6-437a-a4bc-45bb7f499a72 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:41.500605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1143/2286] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:41.630011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9617cc4-0b7d-4a86-9aca-9f9defcb0aaf service swift] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:41.631730 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c9617cc4-0b7d-4a86-9aca-9f9defcb0aaf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:41.739032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9617cc4-0b7d-4a86-9aca-9f9defcb0aaf service swift] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:41.744015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85960212-2ce6-437a-a4bc-45bb7f499a72 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:41.744727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d067f-a291-4582-aeac-30a7b1041d72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:41.745710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-85960212-2ce6-437a-a4bc-45bb7f499a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:41.745869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d067f-a291-4582-aeac-30a7b1041d72 service placement] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:41.746287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d067f-a291-4582-aeac-30a7b1041d72 service placement] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:41.746712 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d067f-a291-4582-aeac-30a7b1041d72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:41.749083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-327d067f-a291-4582-aeac-30a7b1041d72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:41.757591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85960212-2ce6-437a-a4bc-45bb7f499a72 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:41.762010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d06ca-ee69-4470-9056-56ce6911789e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:41.762862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d06ca-ee69-4470-9056-56ce6911789e service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:41.763380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d06ca-ee69-4470-9056-56ce6911789e service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:41.763735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d06ca-ee69-4470-9056-56ce6911789e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:41.766601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5f7d06ca-ee69-4470-9056-56ce6911789e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:41.767885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d067f-a291-4582-aeac-30a7b1041d72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:41.771234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d067f-a291-4582-aeac-30a7b1041d72 service placement] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:41.783071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d06ca-ee69-4470-9056-56ce6911789e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:41.784827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1144/2287] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:41.785118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d06ca-ee69-4470-9056-56ce6911789e service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:41.804719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1144/2288] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:50:43.564114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d067f-a291-4582-aeac-30a7b1041d72 service placement] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:43.575123 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d067f-a291-4582-aeac-30a7b1041d72 service placement] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:43.581919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:43.583034 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:43.583467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:43.583907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] PATH_INFO: `/v3/users/c9990084b9214921bcf24b6d6b34a39c` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:43.585408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9990084b9214921bcf24b6d6b34a39c)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:43.589091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:43.856362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9990084b9214921bcf24b6d6b34a39c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:43.857388 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9990084b9214921bcf24b6d6b34a39c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:43.859243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:43.892658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9990084b9214921bcf24b6d6b34a39c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:43.901398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9990084b9214921bcf24b6d6b34a39c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:44.004158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9990084b9214921bcf24b6d6b34a39c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:44.012962 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1145/2289] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:50:43 2022] DELETE /identity/v3/users/c9990084b9214921bcf24b6d6b34a39c => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:44.017590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c09ca-ea40-4ac4-8d4b-b08791b04ed4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:44.018081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c09ca-ea40-4ac4-8d4b-b08791b04ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:44.018178 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c09ca-ea40-4ac4-8d4b-b08791b04ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:44.045491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-962c09ca-ea40-4ac4-8d4b-b08791b04ed4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:44.046743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-962c09ca-ea40-4ac4-8d4b-b08791b04ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:44.127536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1145/2290] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:44.153516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:44.157314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:44.254374 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99090f7e-ba4e-41db-a047-b444d3439d0e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:44.259789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3d6e-6651-4e28-8c3e-0fc9b4387731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:44.264503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3d6e-6651-4e28-8c3e-0fc9b4387731 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:44.264503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3d6e-6651-4e28-8c3e-0fc9b4387731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:44.264503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3d6e-6651-4e28-8c3e-0fc9b4387731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:44.291122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd3d6e-6651-4e28-8c3e-0fc9b4387731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:44.292891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd3d6e-6651-4e28-8c3e-0fc9b4387731 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:44.307311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1146/2291] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:45.043003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c09ca-ea40-4ac4-8d4b-b08791b04ed4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:45.043473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-962c09ca-ea40-4ac4-8d4b-b08791b04ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:45.056296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c09ca-ea40-4ac4-8d4b-b08791b04ed4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:45.064398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:45.065666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:45.066177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:45.066658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] PATH_INFO: `/v3/projects/0931133a75484c64b26b6c00146e29bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:45.068121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0931133a75484c64b26b6c00146e29bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:45.072041 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:45.230423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0931133a75484c64b26b6c00146e29bd', 'request_id': 'req-84981a88-1a16-4f7e-8fe4-53c0ca716c22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:45.347401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0931133a75484c64b26b6c00146e29bd', 'request_id': 'req-84981a88-1a16-4f7e-8fe4-53c0ca716c22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:45.357305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0931133a75484c64b26b6c00146e29bd', 'request_id': 'req-84981a88-1a16-4f7e-8fe4-53c0ca716c22'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:45.396562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] The token cache is being invalidate because project 0931133a75484c64b26b6c00146e29bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:50:45.397440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:45.398286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-84981a88-1a16-4f7e-8fe4-53c0ca716c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:45.403755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1146/2292] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:50:45 2022] DELETE /identity/v3/projects/0931133a75484c64b26b6c00146e29bd => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:45.405188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5b81-8456-4139-b511-ca1e69d888cd None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:45.405643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5b81-8456-4139-b511-ca1e69d888cd None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:45.405957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5b81-8456-4139-b511-ca1e69d888cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:45.440101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-314e5b81-8456-4139-b511-ca1e69d888cd None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:45.442604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-314e5b81-8456-4139-b511-ca1e69d888cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:45.578755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19dd86-c5f5-434c-ab8f-cecf812e6a22 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:45.579052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19dd86-c5f5-434c-ab8f-cecf812e6a22 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:45.579334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19dd86-c5f5-434c-ab8f-cecf812e6a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:45.583265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1147/2293] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:45.640321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1c19dd86-c5f5-434c-ab8f-cecf812e6a22 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:45.653469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1c19dd86-c5f5-434c-ab8f-cecf812e6a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:45.681547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e5b81-8456-4139-b511-ca1e69d888cd None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:45.688301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1147/2294] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:45.694798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e5b81-8456-4139-b511-ca1e69d888cd None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:45.700979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acfdf8-8afd-4c09-b6ea-f9484ae44373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:45.702096 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acfdf8-8afd-4c09-b6ea-f9484ae44373 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:45.702691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acfdf8-8afd-4c09-b6ea-f9484ae44373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:45.703139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acfdf8-8afd-4c09-b6ea-f9484ae44373 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:45.704082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acfdf8-8afd-4c09-b6ea-f9484ae44373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:50:45.704560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acfdf8-8afd-4c09-b6ea-f9484ae44373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:45.706709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acfdf8-8afd-4c09-b6ea-f9484ae44373 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:45.727891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1148/2295] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:50:45.729206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19dd86-c5f5-434c-ab8f-cecf812e6a22 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:45.742361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19dd86-c5f5-434c-ab8f-cecf812e6a22 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:45.780003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1874e-1982-475f-a5df-b40c2a1703c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:45.781092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1874e-1982-475f-a5df-b40c2a1703c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:45.783717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1874e-1982-475f-a5df-b40c2a1703c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:45.784028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1874e-1982-475f-a5df-b40c2a1703c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:45.785044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1874e-1982-475f-a5df-b40c2a1703c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:45.789747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1874e-1982-475f-a5df-b40c2a1703c6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:45.914702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1148/2296] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:50:45 2022] POST /identity/v3/projects => generated 379 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:50:45.919410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acfdf8-8afd-4c09-b6ea-f9484ae44373 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:45.935627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acfdf8-8afd-4c09-b6ea-f9484ae44373 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:46.007824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:46.018632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4e6f2-c9ce-455f-9568-0dc95e1e9c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:46.019320 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4e6f2-c9ce-455f-9568-0dc95e1e9c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:46.019601 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4e6f2-c9ce-455f-9568-0dc95e1e9c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:46.019877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4e6f2-c9ce-455f-9568-0dc95e1e9c32 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:46.020584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4e6f2-c9ce-455f-9568-0dc95e1e9c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:46.022841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4e6f2-c9ce-455f-9568-0dc95e1e9c32 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:46.128681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1149/2297] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:50:45 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:46.130106 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1874e-1982-475f-a5df-b40c2a1703c6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:46.148773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1874e-1982-475f-a5df-b40c2a1703c6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:46.158955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd11802-5111-4fd0-aab5-d4b07a58dfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:46.159663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd11802-5111-4fd0-aab5-d4b07a58dfda admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:46.162123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd11802-5111-4fd0-aab5-d4b07a58dfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:46.162516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd11802-5111-4fd0-aab5-d4b07a58dfda admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:46.163471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd11802-5111-4fd0-aab5-d4b07a58dfda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:46.165114 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd11802-5111-4fd0-aab5-d4b07a58dfda admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:46.194431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1149/2298] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:46.198253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4e6f2-c9ce-455f-9568-0dc95e1e9c32 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:46.211837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4e6f2-c9ce-455f-9568-0dc95e1e9c32 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:46.230035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae94b46-96b7-46c1-bf8f-5caed7898dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:46.231486 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae94b46-96b7-46c1-bf8f-5caed7898dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:46.232014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae94b46-96b7-46c1-bf8f-5caed7898dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:46.232483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae94b46-96b7-46c1-bf8f-5caed7898dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:46.233526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae94b46-96b7-46c1-bf8f-5caed7898dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:46.236205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae94b46-96b7-46c1-bf8f-5caed7898dbd admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:46.257088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1150/2299] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:46.262718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd11802-5111-4fd0-aab5-d4b07a58dfda admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:46.282767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd11802-5111-4fd0-aab5-d4b07a58dfda admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:46.289998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518425-2dc4-447c-beae-c59286d196d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:46.291022 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518425-2dc4-447c-beae-c59286d196d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:46.291377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518425-2dc4-447c-beae-c59286d196d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:46.291720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518425-2dc4-447c-beae-c59286d196d4 admin admin] PATH_INFO: `/v3/projects/07f2ca8d0ea94adeaf5a5108a7fa4354/users/fa47b48f133b41baaef084ede4fd3931/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:46.321443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:46.329191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56518425-2dc4-447c-beae-c59286d196d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f2ca8d0ea94adeaf5a5108a7fa4354, user_id=fa47b48f133b41baaef084ede4fd3931, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:46.334762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56518425-2dc4-447c-beae-c59286d196d4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:46.426233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1150/2300] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:46 2022] PUT /identity/v3/projects/07f2ca8d0ea94adeaf5a5108a7fa4354/users/fa47b48f133b41baaef084ede4fd3931/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:50:46.436766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5972d-c972-4657-b567-443bc1fa9b11 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:46.437332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5972d-c972-4657-b567-443bc1fa9b11 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:46.437431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5972d-c972-4657-b567-443bc1fa9b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:46.466203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-8cf5972d-c972-4657-b567-443bc1fa9b11 None None] MFA Rules not processed for user `fa47b48f133b41baaef084ede4fd3931`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:46.467358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8cf5972d-c972-4657-b567-443bc1fa9b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:46.520184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518425-2dc4-447c-beae-c59286d196d4 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:46.523064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-56518425-2dc4-447c-beae-c59286d196d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:46.568770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1151/2301] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:46.572877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5972d-c972-4657-b567-443bc1fa9b11 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:46.639384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518425-2dc4-447c-beae-c59286d196d4 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:46.646002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:46.647278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:46.647853 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:46.648467 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] PATH_INFO: `/v3/users/246f07024efa444da74f2b898f426b46` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:46.650614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=246f07024efa444da74f2b898f426b46)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:46.654854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:46.671074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5972d-c972-4657-b567-443bc1fa9b11 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:46.676650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a0a61-e99b-4d25-a575-0550f7e242b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:46.677064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a0a61-e99b-4d25-a575-0550f7e242b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:46.677468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a0a61-e99b-4d25-a575-0550f7e242b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:46.678207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a0a61-e99b-4d25-a575-0550f7e242b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:46.679061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a0a61-e99b-4d25-a575-0550f7e242b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:46.681777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a0a61-e99b-4d25-a575-0550f7e242b3 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:46.788149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1152/2302] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:50:46 2022] POST /identity/v3/projects => generated 381 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:46.792107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a0a61-e99b-4d25-a575-0550f7e242b3 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:46.849462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:46.865692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a0a61-e99b-4d25-a575-0550f7e242b3 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:46.867335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e7d7a-7c1c-4e8b-8623-22584119a970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:46.868015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e7d7a-7c1c-4e8b-8623-22584119a970 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:46.868301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e7d7a-7c1c-4e8b-8623-22584119a970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:46.868579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e7d7a-7c1c-4e8b-8623-22584119a970 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:46.869678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e7d7a-7c1c-4e8b-8623-22584119a970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:46.874707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e7d7a-7c1c-4e8b-8623-22584119a970 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:46.914860 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246f07024efa444da74f2b898f426b46'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:46.923303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246f07024efa444da74f2b898f426b46'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:46.940473 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246f07024efa444da74f2b898f426b46'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:46.974120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1153/2303] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:50:46 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:50:46.975298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e7d7a-7c1c-4e8b-8623-22584119a970 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:46.998765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e7d7a-7c1c-4e8b-8623-22584119a970 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:47.011362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985162d9-ae3a-4c13-92b8-3fb4616d2e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:47.012384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985162d9-ae3a-4c13-92b8-3fb4616d2e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:47.012822 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985162d9-ae3a-4c13-92b8-3fb4616d2e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:47.013281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985162d9-ae3a-4c13-92b8-3fb4616d2e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:47.016534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985162d9-ae3a-4c13-92b8-3fb4616d2e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:47.019212 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985162d9-ae3a-4c13-92b8-3fb4616d2e42 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:47.037964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246f07024efa444da74f2b898f426b46'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:47.038658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246f07024efa444da74f2b898f426b46'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:47.048727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1154/2304] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:47.050049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985162d9-ae3a-4c13-92b8-3fb4616d2e42 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:47.050230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1151/2305] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:50:46 2022] DELETE /identity/v3/users/246f07024efa444da74f2b898f426b46 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:50:47.056878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-985162d9-ae3a-4c13-92b8-3fb4616d2e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:47.084356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985162d9-ae3a-4c13-92b8-3fb4616d2e42 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:47.205029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b4c59-59f2-4096-9efe-3badd0c2be39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:47.205029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b4c59-59f2-4096-9efe-3badd0c2be39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:47.205029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b4c59-59f2-4096-9efe-3badd0c2be39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:47.205029 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b4c59-59f2-4096-9efe-3badd0c2be39 admin admin] PATH_INFO: `/v3/projects/28b9c3595a3d4f66929ac664fd3a9c6b/users/e06f17138e8c4e0c87decd62e3d9d23b/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:47.223190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b4c59-59f2-4096-9efe-3badd0c2be39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b9c3595a3d4f66929ac664fd3a9c6b, user_id=e06f17138e8c4e0c87decd62e3d9d23b, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:47.226166 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b4c59-59f2-4096-9efe-3badd0c2be39 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:47.368241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1155/2306] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:47 2022] PUT /identity/v3/projects/28b9c3595a3d4f66929ac664fd3a9c6b/users/e06f17138e8c4e0c87decd62e3d9d23b/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:47.372179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:47.384606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ccd98-d425-43c0-af78-491c483686fe admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:47.487023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b0522-7014-40c6-80d5-4e9d90903eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:47.487903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b0522-7014-40c6-80d5-4e9d90903eac admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:47.488603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b0522-7014-40c6-80d5-4e9d90903eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:47.489039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b0522-7014-40c6-80d5-4e9d90903eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:47.490080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b0522-7014-40c6-80d5-4e9d90903eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:47.491884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b0522-7014-40c6-80d5-4e9d90903eac admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:47.507785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1152/2307] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:47.512773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b4c59-59f2-4096-9efe-3badd0c2be39 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:47.538988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b4c59-59f2-4096-9efe-3badd0c2be39 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:47.554513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b0522-7014-40c6-80d5-4e9d90903eac admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:47.556536 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-966b0522-7014-40c6-80d5-4e9d90903eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:47.567464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e3ec2-36fc-45f9-924a-139558e97de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:47.569309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e3ec2-36fc-45f9-924a-139558e97de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:47.569857 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e3ec2-36fc-45f9-924a-139558e97de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:47.571193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e3ec2-36fc-45f9-924a-139558e97de5 admin admin] PATH_INFO: `/v3/projects/28b9c3595a3d4f66929ac664fd3a9c6b/users/e06f17138e8c4e0c87decd62e3d9d23b/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:47.576918 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e3ec2-36fc-45f9-924a-139558e97de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b9c3595a3d4f66929ac664fd3a9c6b, user_id=e06f17138e8c4e0c87decd62e3d9d23b, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:47.579742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b0522-7014-40c6-80d5-4e9d90903eac admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:47.583072 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e3ec2-36fc-45f9-924a-139558e97de5 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:47.592637 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:47.594394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:47.594683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:47.594984 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] PATH_INFO: `/v3/projects/48a88ac7db8a481ca03508d315a0bc56` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:47.596872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48a88ac7db8a481ca03508d315a0bc56)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:47.603714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:47.633184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:47.634760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1156/2308] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:47 2022] PUT /identity/v3/projects/28b9c3595a3d4f66929ac664fd3a9c6b/users/e06f17138e8c4e0c87decd62e3d9d23b/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:47.647648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eed14-4c24-4961-8c21-6088bf0818d8 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:47.648088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eed14-4c24-4961-8c21-6088bf0818d8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:47.648465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eed14-4c24-4961-8c21-6088bf0818d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:47.714267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-6e9eed14-4c24-4961-8c21-6088bf0818d8 None None] MFA Rules not processed for user `e06f17138e8c4e0c87decd62e3d9d23b`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:47.715632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6e9eed14-4c24-4961-8c21-6088bf0818d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:47.837135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1157/2309] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:47.849919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9eed14-4c24-4961-8c21-6088bf0818d8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:47.854048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6e9eed14-4c24-4961-8c21-6088bf0818d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:47.875768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48a88ac7db8a481ca03508d315a0bc56', 'request_id': 'req-11dc3a8b-1c1b-4685-a83c-c03d7f781486'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:47.901368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:47.952979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48a88ac7db8a481ca03508d315a0bc56', 'request_id': 'req-11dc3a8b-1c1b-4685-a83c-c03d7f781486'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:47.965105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48a88ac7db8a481ca03508d315a0bc56', 'request_id': 'req-11dc3a8b-1c1b-4685-a83c-c03d7f781486'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:47.972359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9eed14-4c24-4961-8c21-6088bf0818d8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:47.987332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] The token cache is being invalidate because project 48a88ac7db8a481ca03508d315a0bc56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:50:47.987701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:47.988360 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11dc3a8b-1c1b-4685-a83c-c03d7f781486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:47.997729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1153/2310] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:50:47 2022] DELETE /identity/v3/projects/48a88ac7db8a481ca03508d315a0bc56 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:50:47.997729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd457b7-fd9d-46d3-9e22-d40b084804b3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:47.997729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd457b7-fd9d-46d3-9e22-d40b084804b3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:47.997729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd457b7-fd9d-46d3-9e22-d40b084804b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.015906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-edd457b7-fd9d-46d3-9e22-d40b084804b3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:48.017130 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-edd457b7-fd9d-46d3-9e22-d40b084804b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:48.032458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec8e17-df97-4c40-be9d-f8a4e5e87da8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:48.033121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec8e17-df97-4c40-be9d-f8a4e5e87da8 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.033482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec8e17-df97-4c40-be9d-f8a4e5e87da8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.033760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec8e17-df97-4c40-be9d-f8a4e5e87da8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.035665 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f3ec8e17-df97-4c40-be9d-f8a4e5e87da8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:48.121648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1154/2311] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:48.121648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd457b7-fd9d-46d3-9e22-d40b084804b3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:48.121648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-edd457b7-fd9d-46d3-9e22-d40b084804b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:48.133754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd457b7-fd9d-46d3-9e22-d40b084804b3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:48.144415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:48.145705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.147078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.147533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] PATH_INFO: `/v3/users/8773c88f88614a9d903f667305a217e0` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.148990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8773c88f88614a9d903f667305a217e0)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:48.151166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:48.156227 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec8e17-df97-4c40-be9d-f8a4e5e87da8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:48.160325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec8e17-df97-4c40-be9d-f8a4e5e87da8 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:48.192788 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1158/2312] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:50:48.194059 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f174a-7723-4af5-ac24-0de31cfc8d02 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.194406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f174a-7723-4af5-ac24-0de31cfc8d02 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.194689 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f174a-7723-4af5-ac24-0de31cfc8d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.215559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-a09f174a-7723-4af5-ac24-0de31cfc8d02 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:48.216885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a09f174a-7723-4af5-ac24-0de31cfc8d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:48.229272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1159/2313] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:48.233193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f174a-7723-4af5-ac24-0de31cfc8d02 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:48.247910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f174a-7723-4af5-ac24-0de31cfc8d02 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:48.253468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e032de7-b138-4b64-8542-dc47d854bdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:48.254278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e032de7-b138-4b64-8542-dc47d854bdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.254681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e032de7-b138-4b64-8542-dc47d854bdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.255058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e032de7-b138-4b64-8542-dc47d854bdb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.256142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e032de7-b138-4b64-8542-dc47d854bdb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:50:48.256600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e032de7-b138-4b64-8542-dc47d854bdb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:48.258514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e032de7-b138-4b64-8542-dc47d854bdb1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:48.283359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1160/2314] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:50:48.283359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e032de7-b138-4b64-8542-dc47d854bdb1 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:48.297551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e032de7-b138-4b64-8542-dc47d854bdb1 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:48.303126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451ee82-a376-4ecf-80af-6a5dac5b2d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:48.303599 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ee82-a376-4ecf-80af-6a5dac5b2d6c admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.303956 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ee82-a376-4ecf-80af-6a5dac5b2d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.304307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ee82-a376-4ecf-80af-6a5dac5b2d6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.305189 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451ee82-a376-4ecf-80af-6a5dac5b2d6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:48.307277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451ee82-a376-4ecf-80af-6a5dac5b2d6c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:48.385753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8773c88f88614a9d903f667305a217e0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.401457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8773c88f88614a9d903f667305a217e0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.414652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1161/2315] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:50:48 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:48.415917 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451ee82-a376-4ecf-80af-6a5dac5b2d6c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:48.417806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e451ee82-a376-4ecf-80af-6a5dac5b2d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:48.422488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8773c88f88614a9d903f667305a217e0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.496108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451ee82-a376-4ecf-80af-6a5dac5b2d6c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:48.497980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8773c88f88614a9d903f667305a217e0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.503194 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8773c88f88614a9d903f667305a217e0'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.503412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:48.503412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.503412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.503711 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] PATH_INFO: `/v3/users/59ebb8f1e98d40c9aea425fad464f1e3` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.504790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] RBAC: Authorizing `identity:delete_user(user_id=59ebb8f1e98d40c9aea425fad464f1e3)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:48.506106 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1155/2316] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:50:48 2022] DELETE /identity/v3/users/8773c88f88614a9d903f667305a217e0 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:48.506809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:48.507067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:48.508541 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:48.576955 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a5e19-e095-49f0-9220-67f61732e3bb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:48.582459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbb7fd-52c2-4eae-9730-5285c40a672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:48.583214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbb7fd-52c2-4eae-9730-5285c40a672c admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.583568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbb7fd-52c2-4eae-9730-5285c40a672c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.583914 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbb7fd-52c2-4eae-9730-5285c40a672c admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.584686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbb7fd-52c2-4eae-9730-5285c40a672c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:48.586715 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbb7fd-52c2-4eae-9730-5285c40a672c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:48.682468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1156/2317] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:50:48 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:48.682686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59ebb8f1e98d40c9aea425fad464f1e3'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.683132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59ebb8f1e98d40c9aea425fad464f1e3'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.683540 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbb7fd-52c2-4eae-9730-5285c40a672c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:48.685142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dedbb7fd-52c2-4eae-9730-5285c40a672c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:48.696683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59ebb8f1e98d40c9aea425fad464f1e3'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.697991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbb7fd-52c2-4eae-9730-5285c40a672c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:48.703081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c93891-69a2-4b50-b996-bddbe81d3635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:48.703322 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c93891-69a2-4b50-b996-bddbe81d3635 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.704470 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59ebb8f1e98d40c9aea425fad464f1e3'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.704798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c93891-69a2-4b50-b996-bddbe81d3635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.705085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c93891-69a2-4b50-b996-bddbe81d3635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.705867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c93891-69a2-4b50-b996-bddbe81d3635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:48.707462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c93891-69a2-4b50-b996-bddbe81d3635 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:48.718071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1157/2318] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:50:48.723535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c93891-69a2-4b50-b996-bddbe81d3635 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:48.733768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c93891-69a2-4b50-b996-bddbe81d3635 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:48.738521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569da132-d00e-408d-8634-07b2b472b627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:48.739259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569da132-d00e-408d-8634-07b2b472b627 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.739549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569da132-d00e-408d-8634-07b2b472b627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.739831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569da132-d00e-408d-8634-07b2b472b627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.740552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569da132-d00e-408d-8634-07b2b472b627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:48.742171 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569da132-d00e-408d-8634-07b2b472b627 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:48.753689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1158/2319] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:48.754381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569da132-d00e-408d-8634-07b2b472b627 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:48.756099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-569da132-d00e-408d-8634-07b2b472b627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:48.770628 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569da132-d00e-408d-8634-07b2b472b627 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:48.776897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:48.777837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.778153 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.778386 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] PATH_INFO: `/v3/projects/f274d555f5c746cc885394c699b3d767` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.779627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f274d555f5c746cc885394c699b3d767)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:48.781551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:48.801048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59ebb8f1e98d40c9aea425fad464f1e3'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.809436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1162/2320] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:50:48 2022] DELETE /identity/v3/users/59ebb8f1e98d40c9aea425fad464f1e3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:48.810292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:48.897339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9714e-2e6b-4216-899b-5d8bba3b579b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:48.911059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f274d555f5c746cc885394c699b3d767', 'request_id': 'req-b3f29de5-504f-4211-af74-baf86165c67e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:48.968340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c8e22-7006-4fb1-bc0d-1956c29e3972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:48.969600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c8e22-7006-4fb1-bc0d-1956c29e3972 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:48.969961 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c8e22-7006-4fb1-bc0d-1956c29e3972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:48.970308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c8e22-7006-4fb1-bc0d-1956c29e3972 admin admin] PATH_INFO: `/v3/projects/470ff30a8d044d2895f3288850efe24b/users/903e0909e33a4c11add83681440180d2/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:48.988542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c8e22-7006-4fb1-bc0d-1956c29e3972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=470ff30a8d044d2895f3288850efe24b, user_id=903e0909e33a4c11add83681440180d2, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:48.991388 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c8e22-7006-4fb1-bc0d-1956c29e3972 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:49.009936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f274d555f5c746cc885394c699b3d767', 'request_id': 'req-b3f29de5-504f-4211-af74-baf86165c67e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.019296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f274d555f5c746cc885394c699b3d767', 'request_id': 'req-b3f29de5-504f-4211-af74-baf86165c67e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.029650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] The token cache is being invalidate because project f274d555f5c746cc885394c699b3d767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:50:49.030067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.031128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b3f29de5-504f-4211-af74-baf86165c67e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.032729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1159/2321] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:50:48 2022] DELETE /identity/v3/projects/f274d555f5c746cc885394c699b3d767 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:49.037657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a0a55-31fb-46ad-af3d-1bf3d356036a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.037894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a0a55-31fb-46ad-af3d-1bf3d356036a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.038182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a0a55-31fb-46ad-af3d-1bf3d356036a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.057408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-147a0a55-31fb-46ad-af3d-1bf3d356036a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:49.058620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-147a0a55-31fb-46ad-af3d-1bf3d356036a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:49.067237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1160/2322] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:50:49.075034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638b127-55e3-4373-a73d-ab83dcd474f3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.075319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638b127-55e3-4373-a73d-ab83dcd474f3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.075599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638b127-55e3-4373-a73d-ab83dcd474f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.093289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5638b127-55e3-4373-a73d-ab83dcd474f3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:49.094480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5638b127-55e3-4373-a73d-ab83dcd474f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:49.102384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1161/2323] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:50:49.106432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638b127-55e3-4373-a73d-ab83dcd474f3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:49.115875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638b127-55e3-4373-a73d-ab83dcd474f3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:49.120447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bdf3f-4afc-430a-bdc0-786b7aab6d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:49.121074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bdf3f-4afc-430a-bdc0-786b7aab6d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.121398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bdf3f-4afc-430a-bdc0-786b7aab6d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.121653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bdf3f-4afc-430a-bdc0-786b7aab6d47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.122402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bdf3f-4afc-430a-bdc0-786b7aab6d47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:50:49.122809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bdf3f-4afc-430a-bdc0-786b7aab6d47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:49.124510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bdf3f-4afc-430a-bdc0-786b7aab6d47 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:49.139605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1163/2324] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:48 2022] PUT /identity/v3/projects/470ff30a8d044d2895f3288850efe24b/users/903e0909e33a4c11add83681440180d2/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:49.139709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1162/2325] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:50:49.143723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c8e22-7006-4fb1-bc0d-1956c29e3972 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:49.147163 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-212c8e22-7006-4fb1-bc0d-1956c29e3972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:49.151799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc723-a8a0-40db-8b3d-24b18bcd6c68 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.152085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc723-a8a0-40db-8b3d-24b18bcd6c68 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.152515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc723-a8a0-40db-8b3d-24b18bcd6c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.189802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-80ddc723-a8a0-40db-8b3d-24b18bcd6c68 None None] MFA Rules not processed for user `903e0909e33a4c11add83681440180d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:49.190967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-80ddc723-a8a0-40db-8b3d-24b18bcd6c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:49.221236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c8e22-7006-4fb1-bc0d-1956c29e3972 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:49.231141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:49.232061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.232345 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.232624 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] PATH_INFO: `/v3/projects/dc22323158714269b0da2b01121fe3ed` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.233862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc22323158714269b0da2b01121fe3ed)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:49.236574 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:49.261919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1163/2326] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:49.267149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:49.270095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddc723-a8a0-40db-8b3d-24b18bcd6c68 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:49.281435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddc723-a8a0-40db-8b3d-24b18bcd6c68 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:49.286080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894d1f7-c2a2-4a1a-845b-725261eedaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:49.286709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894d1f7-c2a2-4a1a-845b-725261eedaa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.286990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894d1f7-c2a2-4a1a-845b-725261eedaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.287262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894d1f7-c2a2-4a1a-845b-725261eedaa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.287994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894d1f7-c2a2-4a1a-845b-725261eedaa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:49.289936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894d1f7-c2a2-4a1a-845b-725261eedaa0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:49.321261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:49.362560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc22323158714269b0da2b01121fe3ed', 'request_id': 'req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.368841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc22323158714269b0da2b01121fe3ed', 'request_id': 'req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.375052 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc22323158714269b0da2b01121fe3ed', 'request_id': 'req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.377636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1164/2327] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:50:49 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:49.377872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894d1f7-c2a2-4a1a-845b-725261eedaa0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:49.379534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9894d1f7-c2a2-4a1a-845b-725261eedaa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:49.447592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894d1f7-c2a2-4a1a-845b-725261eedaa0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:49.452636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5eec4-4cdd-4e6e-9fdb-332583287703 tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06f17138e8c4e0c87decd62e3d9d23b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28b9c3595a3d4f66929ac664fd3a9c6b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:49.453439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5eec4-4cdd-4e6e-9fdb-332583287703 tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.453737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5eec4-4cdd-4e6e-9fdb-332583287703 tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.454020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5eec4-4cdd-4e6e-9fdb-332583287703 tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.454762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5eec4-4cdd-4e6e-9fdb-332583287703 tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:50:49.455074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5eec4-4cdd-4e6e-9fdb-332583287703 tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:49.455229 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] The token cache is being invalidate because project dc22323158714269b0da2b01121fe3ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:50:49.455482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.456188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.456797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5eec4-4cdd-4e6e-9fdb-332583287703 tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:49.457920 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1164/2328] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:50:49 2022] DELETE /identity/v3/projects/dc22323158714269b0da2b01121fe3ed => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:49.461757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:49.463127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:49.469746 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1165/2329] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:50:49.470578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5eec4-4cdd-4e6e-9fdb-332583287703 tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:49.472353 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-35d5eec4-4cdd-4e6e-9fdb-332583287703 tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:49.562096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5eec4-4cdd-4e6e-9fdb-332583287703 tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:49.566561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e0b0c-e5b1-408f-bf1f-5d650e0de48d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:49.567485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d7f25-4f0d-4759-a1ad-8d24e6dcd912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:49.568169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d7f25-4f0d-4759-a1ad-8d24e6dcd912 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.568480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d7f25-4f0d-4759-a1ad-8d24e6dcd912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.568757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d7f25-4f0d-4759-a1ad-8d24e6dcd912 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.569504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d7f25-4f0d-4759-a1ad-8d24e6dcd912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:49.571485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d7f25-4f0d-4759-a1ad-8d24e6dcd912 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:49.632707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092063fb-dfa0-41d3-a17e-3e9cba8b362b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:49.633390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092063fb-dfa0-41d3-a17e-3e9cba8b362b service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.633675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092063fb-dfa0-41d3-a17e-3e9cba8b362b service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.633948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092063fb-dfa0-41d3-a17e-3e9cba8b362b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.635775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-092063fb-dfa0-41d3-a17e-3e9cba8b362b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:49.665553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:49.672508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1166/2330] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:50:49 2022] POST /identity/v3/users => generated 282 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:49.672508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d7f25-4f0d-4759-a1ad-8d24e6dcd912 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:49.672508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3d3d7f25-4f0d-4759-a1ad-8d24e6dcd912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:49.687054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d7f25-4f0d-4759-a1ad-8d24e6dcd912 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:49.692339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:49.693328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.693603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.693876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] PATH_INFO: `/v3/users/a8b752c6c8f7421786bfa2e8e52619d6` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.695066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8b752c6c8f7421786bfa2e8e52619d6)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:49.697100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:49.697666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092063fb-dfa0-41d3-a17e-3e9cba8b362b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:49.699296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092063fb-dfa0-41d3-a17e-3e9cba8b362b service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:49.709459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1165/2331] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:49.710247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092063fb-dfa0-41d3-a17e-3e9cba8b362b service nova] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:49.711613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-092063fb-dfa0-41d3-a17e-3e9cba8b362b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:49.725325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092063fb-dfa0-41d3-a17e-3e9cba8b362b service nova] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:49.730065 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cf717-7aa1-498e-b4e0-27c44362516c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06f17138e8c4e0c87decd62e3d9d23b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28b9c3595a3d4f66929ac664fd3a9c6b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:49.730690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cf717-7aa1-498e-b4e0-27c44362516c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.730979 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cf717-7aa1-498e-b4e0-27c44362516c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.731352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cf717-7aa1-498e-b4e0-27c44362516c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.731942 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cf717-7aa1-498e-b4e0-27c44362516c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:49.733896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cf717-7aa1-498e-b4e0-27c44362516c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:49.866743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1166/2332] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:50:49 2022] POST /identity/v3/projects => generated 353 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:49.867348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cf717-7aa1-498e-b4e0-27c44362516c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:49.912964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8b752c6c8f7421786bfa2e8e52619d6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.920404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8b752c6c8f7421786bfa2e8e52619d6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.935102 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8b752c6c8f7421786bfa2e8e52619d6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:49.948872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cf717-7aa1-498e-b4e0-27c44362516c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:49.954595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6f626-472a-4d5f-ab8d-24068cc2ce73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:49.955465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6f626-472a-4d5f-ab8d-24068cc2ce73 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:49.955962 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6f626-472a-4d5f-ab8d-24068cc2ce73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:49.956265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6f626-472a-4d5f-ab8d-24068cc2ce73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:49.957012 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6f626-472a-4d5f-ab8d-24068cc2ce73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:49.958887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6f626-472a-4d5f-ab8d-24068cc2ce73 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:49.973555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1167/2333] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:49.973555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6f626-472a-4d5f-ab8d-24068cc2ce73 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:49.977427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:50.038232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6f626-472a-4d5f-ab8d-24068cc2ce73 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:50.043013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de1858-b25b-4989-b89b-dca04170eabc tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06f17138e8c4e0c87decd62e3d9d23b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28b9c3595a3d4f66929ac664fd3a9c6b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:50.043683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de1858-b25b-4989-b89b-dca04170eabc tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:50.043972 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de1858-b25b-4989-b89b-dca04170eabc tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:50.044256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de1858-b25b-4989-b89b-dca04170eabc tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:50.044993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de1858-b25b-4989-b89b-dca04170eabc tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:50:50.045341 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de1858-b25b-4989-b89b-dca04170eabc tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:50.047546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de1858-b25b-4989-b89b-dca04170eabc tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:50.054882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8b752c6c8f7421786bfa2e8e52619d6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.061656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8b752c6c8f7421786bfa2e8e52619d6'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.061762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1168/2334] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:50:50.061762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de1858-b25b-4989-b89b-dca04170eabc tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:50.063756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-63de1858-b25b-4989-b89b-dca04170eabc tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:50.066619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1167/2335] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:50:49 2022] DELETE /identity/v3/users/a8b752c6c8f7421786bfa2e8e52619d6 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:50.084585 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de1858-b25b-4989-b89b-dca04170eabc tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:50.100954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:50.102494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:50.152168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee6142-c694-4ae8-a236-08b5ceb22aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:50.153366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee6142-c694-4ae8-a236-08b5ceb22aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:50.153656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee6142-c694-4ae8-a236-08b5ceb22aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:50.153937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee6142-c694-4ae8-a236-08b5ceb22aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:50.154684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee6142-c694-4ae8-a236-08b5ceb22aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:50.156413 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee6142-c694-4ae8-a236-08b5ceb22aa2 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:50.167116 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1169/2336] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:50.171363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee6142-c694-4ae8-a236-08b5ceb22aa2 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:50.175244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5490666-5e74-4b85-a659-c707207058a5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:50.182411 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06f17138e8c4e0c87decd62e3d9d23b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28b9c3595a3d4f66929ac664fd3a9c6b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:50.183304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:50.183594 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:50.183870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] PATH_INFO: `/v3/projects/f7526d7c6405403ebf7b1669f63e4369` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:50.185115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f7526d7c6405403ebf7b1669f63e4369)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:50.187205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:50.192183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee6142-c694-4ae8-a236-08b5ceb22aa2 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:50.200048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19c876-8974-4ba8-9667-7c1ef22ee0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:50.201289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19c876-8974-4ba8-9667-7c1ef22ee0db admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:50.201581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19c876-8974-4ba8-9667-7c1ef22ee0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:50.202320 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19c876-8974-4ba8-9667-7c1ef22ee0db admin admin] PATH_INFO: `/v3/projects/7f7c1f02b16a479d97d7e55473dbf3a7/users/f6bf2b373bcd455594426033167bbb97/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:50.224946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19c876-8974-4ba8-9667-7c1ef22ee0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7c1f02b16a479d97d7e55473dbf3a7, user_id=f6bf2b373bcd455594426033167bbb97, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:50.233092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19c876-8974-4ba8-9667-7c1ef22ee0db admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:50.388733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1170/2337] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:50 2022] PUT /identity/v3/projects/7f7c1f02b16a479d97d7e55473dbf3a7/users/f6bf2b373bcd455594426033167bbb97/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:50.389809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19c876-8974-4ba8-9667-7c1ef22ee0db admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:50.399586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7526d7c6405403ebf7b1669f63e4369', 'request_id': 'req-fa6d2c62-bf4b-44f2-af77-82727820266c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.400885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-be19c876-8974-4ba8-9667-7c1ef22ee0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:50.466189 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19c876-8974-4ba8-9667-7c1ef22ee0db admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:50.488458 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7526d7c6405403ebf7b1669f63e4369', 'request_id': 'req-fa6d2c62-bf4b-44f2-af77-82727820266c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.489260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:50.494216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7526d7c6405403ebf7b1669f63e4369', 'request_id': 'req-fa6d2c62-bf4b-44f2-af77-82727820266c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.511223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] The token cache is being invalidate because project f7526d7c6405403ebf7b1669f63e4369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:50:50.517673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.517673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fa6d2c62-bf4b-44f2-af77-82727820266c tempest-VolumeQuotaClassesTest-1239833221 tempest-VolumeQuotaClassesTest-1239833221-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.517673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1168/2338] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:50:50 2022] DELETE /identity/v3/projects/f7526d7c6405403ebf7b1669f63e4369 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:50:50.517673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbf0f1-b220-4425-b7c6-eac7966c0e2c None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:50.517673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbf0f1-b220-4425-b7c6-eac7966c0e2c None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:50.518378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbf0f1-b220-4425-b7c6-eac7966c0e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:50.544940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:50.546006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:50.547910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:50.548207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] PATH_INFO: `/v3/projects/3234cee85a5f42018aecf82fff4ad600` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:50.550763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3234cee85a5f42018aecf82fff4ad600)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:50.553484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:50.556584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-81cbf0f1-b220-4425-b7c6-eac7966c0e2c None None] MFA Rules not processed for user `f6bf2b373bcd455594426033167bbb97`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:50.557882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-81cbf0f1-b220-4425-b7c6-eac7966c0e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:50.626855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1169/2339] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:50.631165 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbf0f1-b220-4425-b7c6-eac7966c0e2c None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:50.632619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-81cbf0f1-b220-4425-b7c6-eac7966c0e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:50.646439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbf0f1-b220-4425-b7c6-eac7966c0e2c None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:50.651358 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48a616-91d4-45d6-98e0-06c651737be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:50.652017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48a616-91d4-45d6-98e0-06c651737be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:50.652299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48a616-91d4-45d6-98e0-06c651737be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:50.652570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48a616-91d4-45d6-98e0-06c651737be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:50.653336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48a616-91d4-45d6-98e0-06c651737be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:50.655856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48a616-91d4-45d6-98e0-06c651737be7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:50.684248 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3234cee85a5f42018aecf82fff4ad600', 'request_id': 'req-1ad44660-5170-48c7-8229-3cc8b8e81cbf'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.693758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3234cee85a5f42018aecf82fff4ad600', 'request_id': 'req-1ad44660-5170-48c7-8229-3cc8b8e81cbf'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.700786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3234cee85a5f42018aecf82fff4ad600', 'request_id': 'req-1ad44660-5170-48c7-8229-3cc8b8e81cbf'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.734083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1170/2340] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:50:50 2022] POST /identity/v3/projects => generated 361 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:50:50.736073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48a616-91d4-45d6-98e0-06c651737be7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:50.763276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] The token cache is being invalidate because project 3234cee85a5f42018aecf82fff4ad600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:50:50.763664 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.764265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:50.766006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1171/2341] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:50:50 2022] DELETE /identity/v3/projects/3234cee85a5f42018aecf82fff4ad600 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:50.806223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48a616-91d4-45d6-98e0-06c651737be7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:50.811202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f89981-2199-4515-949c-5c3297301365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:50.812063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f89981-2199-4515-949c-5c3297301365 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:50.812187 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f89981-2199-4515-949c-5c3297301365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:50.812455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f89981-2199-4515-949c-5c3297301365 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:50.813997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f89981-2199-4515-949c-5c3297301365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:50.815994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f89981-2199-4515-949c-5c3297301365 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:50.827494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:50.932361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1171/2342] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:50:50 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:50:50.936708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:50.938336 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:51.015672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad44660-5170-48c7-8229-3cc8b8e81cbf admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:51.020698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13c7ee-dc32-4799-8bec-a3a78c6afbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:51.021400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13c7ee-dc32-4799-8bec-a3a78c6afbd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:51.021991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13c7ee-dc32-4799-8bec-a3a78c6afbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:51.022269 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13c7ee-dc32-4799-8bec-a3a78c6afbd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:51.023001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13c7ee-dc32-4799-8bec-a3a78c6afbd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:51.024604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13c7ee-dc32-4799-8bec-a3a78c6afbd4 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:51.035515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1172/2343] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:51.039896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f89981-2199-4515-949c-5c3297301365 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:51.050641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f89981-2199-4515-949c-5c3297301365 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:51.055493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7d3b-b2a9-4ceb-abe2-1291247f2bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:51.056381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7d3b-b2a9-4ceb-abe2-1291247f2bec admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:51.056679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7d3b-b2a9-4ceb-abe2-1291247f2bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:51.056937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7d3b-b2a9-4ceb-abe2-1291247f2bec admin admin] PATH_INFO: `/v3/projects/75f64634e4a84fbfb4e025508e357e8c/users/aadf6a8bb9a749cc96c18a9c28c66d7d/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:51.071925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d7d3b-b2a9-4ceb-abe2-1291247f2bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f64634e4a84fbfb4e025508e357e8c, user_id=aadf6a8bb9a749cc96c18a9c28c66d7d, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:51.074639 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d7d3b-b2a9-4ceb-abe2-1291247f2bec admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:51.140885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:51.162044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13c7ee-dc32-4799-8bec-a3a78c6afbd4 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:51.163744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-da13c7ee-dc32-4799-8bec-a3a78c6afbd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:51.164871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1172/2344] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:51 2022] PUT /identity/v3/projects/75f64634e4a84fbfb4e025508e357e8c/users/aadf6a8bb9a749cc96c18a9c28c66d7d/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:51.168614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7d3b-b2a9-4ceb-abe2-1291247f2bec admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:51.177636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13c7ee-dc32-4799-8bec-a3a78c6afbd4 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:51.231806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7d3b-b2a9-4ceb-abe2-1291247f2bec admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:51.237257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c3292-1bb7-4037-a682-7a60c4342089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:51.237929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c3292-1bb7-4037-a682-7a60c4342089 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:51.238219 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c3292-1bb7-4037-a682-7a60c4342089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:51.238499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c3292-1bb7-4037-a682-7a60c4342089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:51.238614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:51.239233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c3292-1bb7-4037-a682-7a60c4342089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:51.239428 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:51.239715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:51.240002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] PATH_INFO: `/v3/users/fa47b48f133b41baaef084ede4fd3931` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:51.240903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c3292-1bb7-4037-a682-7a60c4342089 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:51.241280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa47b48f133b41baaef084ede4fd3931)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:51.251111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:51.254552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1173/2345] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:50:51.255599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c3292-1bb7-4037-a682-7a60c4342089 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:51.266325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c3292-1bb7-4037-a682-7a60c4342089 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:51.271077 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bb4b5-174b-41fc-9de5-5b58bbbe9011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:51.272235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bb4b5-174b-41fc-9de5-5b58bbbe9011 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:51.272520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bb4b5-174b-41fc-9de5-5b58bbbe9011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:51.272797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bb4b5-174b-41fc-9de5-5b58bbbe9011 admin admin] PATH_INFO: `/v3/projects/75f64634e4a84fbfb4e025508e357e8c/users/aadf6a8bb9a749cc96c18a9c28c66d7d/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:51.274934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bb4b5-174b-41fc-9de5-5b58bbbe9011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f64634e4a84fbfb4e025508e357e8c, user_id=aadf6a8bb9a749cc96c18a9c28c66d7d, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:51.277666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bb4b5-174b-41fc-9de5-5b58bbbe9011 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:51.288596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1174/2346] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:51 2022] PUT /identity/v3/projects/75f64634e4a84fbfb4e025508e357e8c/users/aadf6a8bb9a749cc96c18a9c28c66d7d/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:51.298439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5f08c-ec20-4b9e-9f11-8b819df4bfd3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:51.298730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5f08c-ec20-4b9e-9f11-8b819df4bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:51.299010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5f08c-ec20-4b9e-9f11-8b819df4bfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:51.326022 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-bbb5f08c-ec20-4b9e-9f11-8b819df4bfd3 None None] MFA Rules not processed for user `aadf6a8bb9a749cc96c18a9c28c66d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:51.327239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bbb5f08c-ec20-4b9e-9f11-8b819df4bfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:51.394680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1175/2347] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:51 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:50:51.407839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5f08c-ec20-4b9e-9f11-8b819df4bfd3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:51.409340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bbb5f08c-ec20-4b9e-9f11-8b819df4bfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:51.464232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5f08c-ec20-4b9e-9f11-8b819df4bfd3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:51.468069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a66d91-3573-4353-92cc-225fd57bb0de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:51.468726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a66d91-3573-4353-92cc-225fd57bb0de service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:51.469707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a66d91-3573-4353-92cc-225fd57bb0de service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:51.469903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a66d91-3573-4353-92cc-225fd57bb0de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:51.478470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:51.481631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa47b48f133b41baaef084ede4fd3931'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:51.482377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a66d91-3573-4353-92cc-225fd57bb0de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:51.482501 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa47b48f133b41baaef084ede4fd3931'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:51.483822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a66d91-3573-4353-92cc-225fd57bb0de service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:51.493725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1176/2348] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:51 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:50:51.495659 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa47b48f133b41baaef084ede4fd3931'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:51.503082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa47b48f133b41baaef084ede4fd3931'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:51.593273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:51.594600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa47b48f133b41baaef084ede4fd3931'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:51.602834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1173/2349] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:50:51 2022] DELETE /identity/v3/users/fa47b48f133b41baaef084ede4fd3931 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:50:51.607144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d3c1d-6e5c-4f36-83a4-62df8d489406 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:51.607442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d3c1d-6e5c-4f36-83a4-62df8d489406 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:51.607765 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d3c1d-6e5c-4f36-83a4-62df8d489406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:51.627112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f69d3c1d-6e5c-4f36-83a4-62df8d489406 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:51.630993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f69d3c1d-6e5c-4f36-83a4-62df8d489406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:51.689400 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1177/2350] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:50:51.699858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:51.700111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:51.734938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d3c1d-6e5c-4f36-83a4-62df8d489406 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:51.736554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f69d3c1d-6e5c-4f36-83a4-62df8d489406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:51.762768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22418-f17f-4b99-9f9a-73e60c8bb39b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:51.768710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920548f-5cf5-4f21-a510-9520eaa659ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:51.768710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920548f-5cf5-4f21-a510-9520eaa659ed service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:51.768710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920548f-5cf5-4f21-a510-9520eaa659ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:51.768710 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920548f-5cf5-4f21-a510-9520eaa659ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:51.778688 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920548f-5cf5-4f21-a510-9520eaa659ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:51.780145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920548f-5cf5-4f21-a510-9520eaa659ed service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:51.791364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1174/2351] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:51.799405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d3c1d-6e5c-4f36-83a4-62df8d489406 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:51.803387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7d415-4ec0-4d04-9996-e5c229500e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:51.804483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7d415-4ec0-4d04-9996-e5c229500e8d service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:51.804483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7d415-4ec0-4d04-9996-e5c229500e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:51.804756 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7d415-4ec0-4d04-9996-e5c229500e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:51.806647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-69c7d415-4ec0-4d04-9996-e5c229500e8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:51.889530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7d415-4ec0-4d04-9996-e5c229500e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:51.893032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7d415-4ec0-4d04-9996-e5c229500e8d service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:51.906987 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1178/2352] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:50:52.285795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920548f-5cf5-4f21-a510-9520eaa659ed service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:52.287279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3920548f-5cf5-4f21-a510-9520eaa659ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:52.300656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920548f-5cf5-4f21-a510-9520eaa659ed service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:52.307878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:52.308831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:52.309102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:52.309507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] PATH_INFO: `/v3/projects/07f2ca8d0ea94adeaf5a5108a7fa4354` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:52.310965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f2ca8d0ea94adeaf5a5108a7fa4354)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:52.313038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:52.407377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:52.432714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f2ca8d0ea94adeaf5a5108a7fa4354', 'request_id': 'req-b48145f3-75e2-4efd-b039-17af12d47de0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:52.445338 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f2ca8d0ea94adeaf5a5108a7fa4354', 'request_id': 'req-b48145f3-75e2-4efd-b039-17af12d47de0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:52.454348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f2ca8d0ea94adeaf5a5108a7fa4354', 'request_id': 'req-b48145f3-75e2-4efd-b039-17af12d47de0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:52.521428 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] The token cache is being invalidate because project 07f2ca8d0ea94adeaf5a5108a7fa4354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:50:52.521989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:52.522658 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:52.524765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1175/2353] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:50:52 2022] DELETE /identity/v3/projects/07f2ca8d0ea94adeaf5a5108a7fa4354 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:52.528462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7d415-4ec0-4d04-9996-e5c229500e8d service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:52.540202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7d415-4ec0-4d04-9996-e5c229500e8d service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:52.580024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:52.597747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:52.598625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:52.598938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:52.599216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] PATH_INFO: `/v3/users/e06f17138e8c4e0c87decd62e3d9d23b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:52.600419 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e06f17138e8c4e0c87decd62e3d9d23b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:52.602373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:52.834418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e06f17138e8c4e0c87decd62e3d9d23b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:52.841178 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e06f17138e8c4e0c87decd62e3d9d23b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:52.853521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e06f17138e8c4e0c87decd62e3d9d23b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:52.924570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e06f17138e8c4e0c87decd62e3d9d23b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:52.925210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-58756487-07a0-4c2a-8b21-9151085ee37e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e06f17138e8c4e0c87decd62e3d9d23b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:52.932000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1179/2354] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:50:52 2022] DELETE /identity/v3/users/e06f17138e8c4e0c87decd62e3d9d23b => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:53.077870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:53.215425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:53.425559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:53.427077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:53.441491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48145f3-75e2-4efd-b039-17af12d47de0 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:53.472730 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:53.523041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:53.524072 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:53.524882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:53.525181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] PATH_INFO: `/v3/projects/28b9c3595a3d4f66929ac664fd3a9c6b` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:53.526635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b9c3595a3d4f66929ac664fd3a9c6b)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:53.529219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:53.635932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b9c3595a3d4f66929ac664fd3a9c6b', 'request_id': 'req-da68516b-a0cf-41ef-993f-f5dcb786c996'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:53.642795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b9c3595a3d4f66929ac664fd3a9c6b', 'request_id': 'req-da68516b-a0cf-41ef-993f-f5dcb786c996'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:53.648891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b9c3595a3d4f66929ac664fd3a9c6b', 'request_id': 'req-da68516b-a0cf-41ef-993f-f5dcb786c996'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:53.731934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] The token cache is being invalidate because project 28b9c3595a3d4f66929ac664fd3a9c6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:50:53.732132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:53.732771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-da68516b-a0cf-41ef-993f-f5dcb786c996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:53.734607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1176/2355] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:50:53 2022] DELETE /identity/v3/projects/28b9c3595a3d4f66929ac664fd3a9c6b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:53.739320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93274203-1d77-4278-b429-2f462754a8cd None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:53.739817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93274203-1d77-4278-b429-2f462754a8cd None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:53.739935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93274203-1d77-4278-b429-2f462754a8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:53.757615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-93274203-1d77-4278-b429-2f462754a8cd None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:53.758816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-93274203-1d77-4278-b429-2f462754a8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:53.818162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1180/2356] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:53.827902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16284d-75de-40d6-aa9b-1fc574a3e0d3 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:53.833954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16284d-75de-40d6-aa9b-1fc574a3e0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:53.833954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16284d-75de-40d6-aa9b-1fc574a3e0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:53.849591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-aa16284d-75de-40d6-aa9b-1fc574a3e0d3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:53.850800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-aa16284d-75de-40d6-aa9b-1fc574a3e0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:53.858908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1177/2357] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:53.863522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93274203-1d77-4278-b429-2f462754a8cd None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:53.873410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93274203-1d77-4278-b429-2f462754a8cd None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:53.878669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e7bc5-b07b-414e-a1f3-511d9a2feb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:53.879325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e7bc5-b07b-414e-a1f3-511d9a2feb98 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:53.879606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e7bc5-b07b-414e-a1f3-511d9a2feb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:53.879876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e7bc5-b07b-414e-a1f3-511d9a2feb98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:53.880609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e7bc5-b07b-414e-a1f3-511d9a2feb98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:50:53.880962 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e7bc5-b07b-414e-a1f3-511d9a2feb98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:53.889396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e7bc5-b07b-414e-a1f3-511d9a2feb98 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:53.902499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1181/2358] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:50:53.907051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16284d-75de-40d6-aa9b-1fc574a3e0d3 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:53.916168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16284d-75de-40d6-aa9b-1fc574a3e0d3 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:53.920809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1bd5a-e006-40f6-bf82-b3abb4f86dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:53.921493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1bd5a-e006-40f6-bf82-b3abb4f86dcb admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:53.921772 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1bd5a-e006-40f6-bf82-b3abb4f86dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:53.926472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1bd5a-e006-40f6-bf82-b3abb4f86dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:53.926472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1bd5a-e006-40f6-bf82-b3abb4f86dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:53.926472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1bd5a-e006-40f6-bf82-b3abb4f86dcb admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:54.100202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1178/2359] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:50:53 2022] POST /identity/v3/projects => generated 393 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:54.104521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e7bc5-b07b-414e-a1f3-511d9a2feb98 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:54.114707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e7bc5-b07b-414e-a1f3-511d9a2feb98 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:54.172563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66be21-228e-49a2-ab7c-e1638e735189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:54.173248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66be21-228e-49a2-ab7c-e1638e735189 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:54.173528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66be21-228e-49a2-ab7c-e1638e735189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:54.173799 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66be21-228e-49a2-ab7c-e1638e735189 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:54.174481 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66be21-228e-49a2-ab7c-e1638e735189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:54.176371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66be21-228e-49a2-ab7c-e1638e735189 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:54.341625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1182/2360] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:50:54 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:54.355813 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1bd5a-e006-40f6-bf82-b3abb4f86dcb admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:54.366649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1bd5a-e006-40f6-bf82-b3abb4f86dcb admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:54.371280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417a1d9-1df2-4fa6-811e-9e6ac0aeb2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:54.371950 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417a1d9-1df2-4fa6-811e-9e6ac0aeb2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:54.372232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417a1d9-1df2-4fa6-811e-9e6ac0aeb2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:54.372508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417a1d9-1df2-4fa6-811e-9e6ac0aeb2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:54.373392 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417a1d9-1df2-4fa6-811e-9e6ac0aeb2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:54.375023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417a1d9-1df2-4fa6-811e-9e6ac0aeb2c9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:54.389412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1179/2361] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:54.393555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66be21-228e-49a2-ab7c-e1638e735189 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:54.405589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66be21-228e-49a2-ab7c-e1638e735189 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:54.410822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad4e9a-d283-4909-a619-3ce08724ca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:54.411743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4e9a-d283-4909-a619-3ce08724ca74 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:54.411864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4e9a-d283-4909-a619-3ce08724ca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:54.412037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4e9a-d283-4909-a619-3ce08724ca74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:54.412795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad4e9a-d283-4909-a619-3ce08724ca74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:54.414440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad4e9a-d283-4909-a619-3ce08724ca74 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:54.427287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1183/2362] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:54.431484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417a1d9-1df2-4fa6-811e-9e6ac0aeb2c9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:54.442741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417a1d9-1df2-4fa6-811e-9e6ac0aeb2c9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:54.447471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0d939-d726-4261-ac71-1ff54b6599f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:54.448354 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0d939-d726-4261-ac71-1ff54b6599f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:54.448675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0d939-d726-4261-ac71-1ff54b6599f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:54.448945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0d939-d726-4261-ac71-1ff54b6599f8 admin admin] PATH_INFO: `/v3/projects/04a48027848d4de7bd9e20c1fae32ae1/users/db5d9b69b93a439d849cc3fd2a4ef4e4/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:54.463800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0d939-d726-4261-ac71-1ff54b6599f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a48027848d4de7bd9e20c1fae32ae1, user_id=db5d9b69b93a439d849cc3fd2a4ef4e4, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:54.466528 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0d939-d726-4261-ac71-1ff54b6599f8 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:54.538759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1180/2363] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:54 2022] PUT /identity/v3/projects/04a48027848d4de7bd9e20c1fae32ae1/users/db5d9b69b93a439d849cc3fd2a4ef4e4/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:54.549558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc0142-b5cf-4335-962a-5006fce4bed8 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:54.549848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc0142-b5cf-4335-962a-5006fce4bed8 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:54.550128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc0142-b5cf-4335-962a-5006fce4bed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:54.578259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c3bc0142-b5cf-4335-962a-5006fce4bed8 None None] MFA Rules not processed for user `db5d9b69b93a439d849cc3fd2a4ef4e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:54.579448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c3bc0142-b5cf-4335-962a-5006fce4bed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:54.611805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:54.643436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1184/2364] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:54.648452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0d939-d726-4261-ac71-1ff54b6599f8 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:54.695944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:54.709889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0d939-d726-4261-ac71-1ff54b6599f8 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:54.714923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6db0a1-dc76-4065-bd71-3c477b02060c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:54.715591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db0a1-dc76-4065-bd71-3c477b02060c admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:54.715880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db0a1-dc76-4065-bd71-3c477b02060c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:54.716190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db0a1-dc76-4065-bd71-3c477b02060c admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:54.716899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6db0a1-dc76-4065-bd71-3c477b02060c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:54.719380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6db0a1-dc76-4065-bd71-3c477b02060c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:54.796437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1181/2365] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:50:54 2022] POST /identity/v3/projects => generated 393 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:54.801239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc0142-b5cf-4335-962a-5006fce4bed8 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:54.811932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc0142-b5cf-4335-962a-5006fce4bed8 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:54.867217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f357375-1d0b-4729-9756-706e340ed29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:54.867830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357375-1d0b-4729-9756-706e340ed29e admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:54.868075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357375-1d0b-4729-9756-706e340ed29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:54.868346 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357375-1d0b-4729-9756-706e340ed29e admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:54.869120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f357375-1d0b-4729-9756-706e340ed29e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:54.870966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f357375-1d0b-4729-9756-706e340ed29e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:54.973151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1185/2366] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:50:54 2022] POST /identity/v3/users => generated 303 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:50:54.977406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6db0a1-dc76-4065-bd71-3c477b02060c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:54.988232 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6db0a1-dc76-4065-bd71-3c477b02060c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:54.993089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6c9c1-665e-4fd6-89bc-d8aa266d7ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:54.993785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6c9c1-665e-4fd6-89bc-d8aa266d7ace admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:54.994071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6c9c1-665e-4fd6-89bc-d8aa266d7ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:54.994351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6c9c1-665e-4fd6-89bc-d8aa266d7ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:54.995222 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6c9c1-665e-4fd6-89bc-d8aa266d7ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:54.996967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6c9c1-665e-4fd6-89bc-d8aa266d7ace admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:55.008012 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1182/2367] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:55.012241 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f357375-1d0b-4729-9756-706e340ed29e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:55.023195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f357375-1d0b-4729-9756-706e340ed29e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:55.028053 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521cfc8-c636-4456-9a9a-f96f0fcf181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:55.028928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521cfc8-c636-4456-9a9a-f96f0fcf181f admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:55.029244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521cfc8-c636-4456-9a9a-f96f0fcf181f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:55.029524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521cfc8-c636-4456-9a9a-f96f0fcf181f admin admin] PATH_INFO: `/v3/projects/90aca639f1364d11a57f3ea86f1a90c0/users/451504e8fb124182b05b8902299ed767/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:55.045820 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521cfc8-c636-4456-9a9a-f96f0fcf181f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90aca639f1364d11a57f3ea86f1a90c0, user_id=451504e8fb124182b05b8902299ed767, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:55.048079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521cfc8-c636-4456-9a9a-f96f0fcf181f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:55.145075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1186/2368] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:55 2022] PUT /identity/v3/projects/90aca639f1364d11a57f3ea86f1a90c0/users/451504e8fb124182b05b8902299ed767/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:50:55.149357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6c9c1-665e-4fd6-89bc-d8aa266d7ace admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:55.160241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6c9c1-665e-4fd6-89bc-d8aa266d7ace admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:55.215813 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5e5ec-a130-4efb-a0da-8e3c1379abb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:55.216476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5e5ec-a130-4efb-a0da-8e3c1379abb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:55.216761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5e5ec-a130-4efb-a0da-8e3c1379abb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:55.217039 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5e5ec-a130-4efb-a0da-8e3c1379abb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:55.217799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5e5ec-a130-4efb-a0da-8e3c1379abb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:55.220061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5e5ec-a130-4efb-a0da-8e3c1379abb4 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:55.229798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1183/2369] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:50:55.234879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521cfc8-c636-4456-9a9a-f96f0fcf181f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:55.246779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521cfc8-c636-4456-9a9a-f96f0fcf181f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:55.251580 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24074c-c991-4cde-afbb-9c2de06670db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:55.252389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24074c-c991-4cde-afbb-9c2de06670db admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:55.252672 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24074c-c991-4cde-afbb-9c2de06670db admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:55.252947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24074c-c991-4cde-afbb-9c2de06670db admin admin] PATH_INFO: `/v3/projects/90aca639f1364d11a57f3ea86f1a90c0/users/451504e8fb124182b05b8902299ed767/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:55.254941 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24074c-c991-4cde-afbb-9c2de06670db admin admin] RBAC: Authorizing `identity:create_grant(project_id=90aca639f1364d11a57f3ea86f1a90c0, user_id=451504e8fb124182b05b8902299ed767, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:55.257686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24074c-c991-4cde-afbb-9c2de06670db admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:55.268901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1187/2370] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:50:55 2022] PUT /identity/v3/projects/90aca639f1364d11a57f3ea86f1a90c0/users/451504e8fb124182b05b8902299ed767/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:50:55.279848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5697a4-0dea-4486-8830-acf2cfe781a4 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:55.280134 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5697a4-0dea-4486-8830-acf2cfe781a4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:55.282064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5697a4-0dea-4486-8830-acf2cfe781a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:55.308733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-cf5697a4-0dea-4486-8830-acf2cfe781a4 None None] MFA Rules not processed for user `451504e8fb124182b05b8902299ed767`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:55.309991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cf5697a4-0dea-4486-8830-acf2cfe781a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:55.386376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1184/2371] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:55.397010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24074c-c991-4cde-afbb-9c2de06670db admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:55.399075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8e24074c-c991-4cde-afbb-9c2de06670db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:55.457477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24074c-c991-4cde-afbb-9c2de06670db admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:55.461123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863f5c6-d9df-46bd-b8e2-4bb081174c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:55.461795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863f5c6-d9df-46bd-b8e2-4bb081174c5f service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:55.462078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863f5c6-d9df-46bd-b8e2-4bb081174c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:55.462352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863f5c6-d9df-46bd-b8e2-4bb081174c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:55.471401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863f5c6-d9df-46bd-b8e2-4bb081174c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:55.472822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863f5c6-d9df-46bd-b8e2-4bb081174c5f service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:55.482275 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1188/2372] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:55.530762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5697a4-0dea-4486-8830-acf2cfe781a4 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:55.532426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cf5697a4-0dea-4486-8830-acf2cfe781a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:55.545231 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5697a4-0dea-4486-8830-acf2cfe781a4 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:55.548994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19e413-c83f-427b-bf84-72f880c5a0ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:55.549580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19e413-c83f-427b-bf84-72f880c5a0ec service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:55.549863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19e413-c83f-427b-bf84-72f880c5a0ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:55.552848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19e413-c83f-427b-bf84-72f880c5a0ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:55.597978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:55.616267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19e413-c83f-427b-bf84-72f880c5a0ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:55.617797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19e413-c83f-427b-bf84-72f880c5a0ec service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:55.628130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1185/2373] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:58.405710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863f5c6-d9df-46bd-b8e2-4bb081174c5f service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:58.407173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7863f5c6-d9df-46bd-b8e2-4bb081174c5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:58.440653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:58.474175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863f5c6-d9df-46bd-b8e2-4bb081174c5f service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:58.478957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:58.479883 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:58.480164 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:58.480437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] PATH_INFO: `/v3/users/903e0909e33a4c11add83681440180d2` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:58.481664 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=903e0909e33a4c11add83681440180d2)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:58.483569 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:58.658106 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '903e0909e33a4c11add83681440180d2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:58.664915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '903e0909e33a4c11add83681440180d2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:58.676932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '903e0909e33a4c11add83681440180d2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:58.749567 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '903e0909e33a4c11add83681440180d2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:58.763950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '903e0909e33a4c11add83681440180d2'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:58.763950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1189/2374] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:50:58 2022] DELETE /identity/v3/users/903e0909e33a4c11add83681440180d2 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:58.764148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484f30d-346a-4f36-b021-3bd3fecbce35 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:58.764577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484f30d-346a-4f36-b021-3bd3fecbce35 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:58.764666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484f30d-346a-4f36-b021-3bd3fecbce35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:58.784274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-0484f30d-346a-4f36-b021-3bd3fecbce35 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:58.785497 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0484f30d-346a-4f36-b021-3bd3fecbce35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:58.848872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1186/2375] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:58.858542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:58.859951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:58.920032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecad860-16d2-45f9-9966-47d3e5bc35c8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:58.923760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62011857-df42-4b73-aa75-79a33a7fe652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:58.924915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62011857-df42-4b73-aa75-79a33a7fe652 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:58.925223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62011857-df42-4b73-aa75-79a33a7fe652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:58.925499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62011857-df42-4b73-aa75-79a33a7fe652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:58.934966 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62011857-df42-4b73-aa75-79a33a7fe652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:58.936424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62011857-df42-4b73-aa75-79a33a7fe652 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:58.946530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1190/2376] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:50:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:50:59.319415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484f30d-346a-4f36-b021-3bd3fecbce35 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:59.320840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0484f30d-346a-4f36-b021-3bd3fecbce35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:59.333149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484f30d-346a-4f36-b021-3bd3fecbce35 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:59.337944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:59.340126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:59.340126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:59.340126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] PATH_INFO: `/v3/projects/470ff30a8d044d2895f3288850efe24b` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:59.340539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=470ff30a8d044d2895f3288850efe24b)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:59.342567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:59.473316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470ff30a8d044d2895f3288850efe24b', 'request_id': 'req-5611cbe7-9057-467a-b7f1-0cd905352dfa'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:59.479740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470ff30a8d044d2895f3288850efe24b', 'request_id': 'req-5611cbe7-9057-467a-b7f1-0cd905352dfa'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:59.485838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470ff30a8d044d2895f3288850efe24b', 'request_id': 'req-5611cbe7-9057-467a-b7f1-0cd905352dfa'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:59.612410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] The token cache is being invalidate because project 470ff30a8d044d2895f3288850efe24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:50:59.612622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:59.613352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5611cbe7-9057-467a-b7f1-0cd905352dfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:50:59.614888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1187/2377] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:50:59 2022] DELETE /identity/v3/projects/470ff30a8d044d2895f3288850efe24b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:50:59.620753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f1dc3-aa8a-4d6a-8f76-3b2d10fe28e6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:59.621260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f1dc3-aa8a-4d6a-8f76-3b2d10fe28e6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:59.621406 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f1dc3-aa8a-4d6a-8f76-3b2d10fe28e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:59.638620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a51f1dc3-aa8a-4d6a-8f76-3b2d10fe28e6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:59.639806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a51f1dc3-aa8a-4d6a-8f76-3b2d10fe28e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:59.683720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:50:59.697182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1191/2378] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:59.706325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f69e7-e4f1-4263-94be-0689624e0b1c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:59.706612 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f69e7-e4f1-4263-94be-0689624e0b1c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:59.706890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f69e7-e4f1-4263-94be-0689624e0b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:59.725956 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-a46f69e7-e4f1-4263-94be-0689624e0b1c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:50:59.727208 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a46f69e7-e4f1-4263-94be-0689624e0b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:50:59.735270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1188/2379] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:50:59.739553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f1dc3-aa8a-4d6a-8f76-3b2d10fe28e6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:59.748891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f1dc3-aa8a-4d6a-8f76-3b2d10fe28e6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:59.753376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c4e3e-7e67-4fee-b643-02f09fc92f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:59.754029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c4e3e-7e67-4fee-b643-02f09fc92f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:59.754513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c4e3e-7e67-4fee-b643-02f09fc92f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:59.754958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c4e3e-7e67-4fee-b643-02f09fc92f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:59.756050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c4e3e-7e67-4fee-b643-02f09fc92f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:50:59.756513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c4e3e-7e67-4fee-b643-02f09fc92f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:59.758121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c4e3e-7e67-4fee-b643-02f09fc92f7b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:59.773202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1192/2380] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:50:59.775267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f69e7-e4f1-4263-94be-0689624e0b1c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:59.785369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f69e7-e4f1-4263-94be-0689624e0b1c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:59.789803 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63b17f-fc0d-4439-9827-7e25d54ff557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:59.790446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63b17f-fc0d-4439-9827-7e25d54ff557 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:59.790729 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63b17f-fc0d-4439-9827-7e25d54ff557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:59.791001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63b17f-fc0d-4439-9827-7e25d54ff557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:59.791701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63b17f-fc0d-4439-9827-7e25d54ff557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:59.793633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63b17f-fc0d-4439-9827-7e25d54ff557 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:50:59.886678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1189/2381] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:50:59 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:50:59.891174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c4e3e-7e67-4fee-b643-02f09fc92f7b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:50:59.902057 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c4e3e-7e67-4fee-b643-02f09fc92f7b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:50:59.959875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26cf9e-885d-4b14-a3d1-0ecc2be11d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:50:59.960544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26cf9e-885d-4b14-a3d1-0ecc2be11d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:50:59.960831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26cf9e-885d-4b14-a3d1-0ecc2be11d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:50:59.961105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26cf9e-885d-4b14-a3d1-0ecc2be11d0f admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:50:59.961826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26cf9e-885d-4b14-a3d1-0ecc2be11d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:50:59.963743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26cf9e-885d-4b14-a3d1-0ecc2be11d0f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:00.035245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1193/2382] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:50:59 2022] POST /identity/v3/users => generated 291 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:00.039475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63b17f-fc0d-4439-9827-7e25d54ff557 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:00.051355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63b17f-fc0d-4439-9827-7e25d54ff557 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:00.056121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7d484-09e9-436e-9411-c7570f02275c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:00.060414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7d484-09e9-436e-9411-c7570f02275c admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:00.060414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7d484-09e9-436e-9411-c7570f02275c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:00.060414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7d484-09e9-436e-9411-c7570f02275c admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:00.060414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7d484-09e9-436e-9411-c7570f02275c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:00.060414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7d484-09e9-436e-9411-c7570f02275c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:00.071167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1190/2383] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:51:00.075232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26cf9e-885d-4b14-a3d1-0ecc2be11d0f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:00.085814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26cf9e-885d-4b14-a3d1-0ecc2be11d0f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:00.090564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd89cb9-a358-4ede-8c2d-90fa155b4892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:00.091234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd89cb9-a358-4ede-8c2d-90fa155b4892 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:00.091539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd89cb9-a358-4ede-8c2d-90fa155b4892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:00.091887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd89cb9-a358-4ede-8c2d-90fa155b4892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:00.092568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd89cb9-a358-4ede-8c2d-90fa155b4892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:00.094205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd89cb9-a358-4ede-8c2d-90fa155b4892 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:00.104610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1194/2384] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:00.108745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7d484-09e9-436e-9411-c7570f02275c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:00.118908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7d484-09e9-436e-9411-c7570f02275c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:00.123679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1dfff-c445-4496-a391-2e3580b4caba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:00.124552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1dfff-c445-4496-a391-2e3580b4caba admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:00.124838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1dfff-c445-4496-a391-2e3580b4caba admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:00.125117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1dfff-c445-4496-a391-2e3580b4caba admin admin] PATH_INFO: `/v3/projects/7a7a7f2803f54dd3b700177a632f4147/users/d4782516924347b68f86b0c56ac36d37/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:00.140421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1dfff-c445-4496-a391-2e3580b4caba admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7a7f2803f54dd3b700177a632f4147, user_id=d4782516924347b68f86b0c56ac36d37, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:00.143761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1dfff-c445-4496-a391-2e3580b4caba admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:00.216608 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:00.221379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1191/2385] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:00 2022] PUT /identity/v3/projects/7a7a7f2803f54dd3b700177a632f4147/users/d4782516924347b68f86b0c56ac36d37/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:00.233448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe566ae8-4ab3-4c88-911e-df28ab89dd20 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:00.233744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe566ae8-4ab3-4c88-911e-df28ab89dd20 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:00.234029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe566ae8-4ab3-4c88-911e-df28ab89dd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:00.263824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fe566ae8-4ab3-4c88-911e-df28ab89dd20 None None] MFA Rules not processed for user `d4782516924347b68f86b0c56ac36d37`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:00.265120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fe566ae8-4ab3-4c88-911e-df28ab89dd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:00.333381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1195/2386] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:00.337867 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1dfff-c445-4496-a391-2e3580b4caba admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:00.406438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1dfff-c445-4496-a391-2e3580b4caba admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:00.411278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08b9ff-de48-4dc2-a772-789f61dbacb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:00.411951 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08b9ff-de48-4dc2-a772-789f61dbacb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:00.412235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08b9ff-de48-4dc2-a772-789f61dbacb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:00.412510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08b9ff-de48-4dc2-a772-789f61dbacb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:00.413275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08b9ff-de48-4dc2-a772-789f61dbacb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:00.415207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08b9ff-de48-4dc2-a772-789f61dbacb4 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:00.509111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1192/2387] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:00 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:00.513021 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe566ae8-4ab3-4c88-911e-df28ab89dd20 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:00.524200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe566ae8-4ab3-4c88-911e-df28ab89dd20 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:00.569516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:00.582287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd46600-9b7a-4bf4-92cd-e5bdd78287be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:00.582964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd46600-9b7a-4bf4-92cd-e5bdd78287be admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:00.583243 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd46600-9b7a-4bf4-92cd-e5bdd78287be admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:00.583514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd46600-9b7a-4bf4-92cd-e5bdd78287be admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:00.584223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd46600-9b7a-4bf4-92cd-e5bdd78287be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:00.586273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd46600-9b7a-4bf4-92cd-e5bdd78287be admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:00.684602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1196/2388] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:00 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:00.688410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08b9ff-de48-4dc2-a772-789f61dbacb4 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:00.700127 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08b9ff-de48-4dc2-a772-789f61dbacb4 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:00.705589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1cb6d-36b4-41b9-9488-c03df3d69350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:00.706308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1cb6d-36b4-41b9-9488-c03df3d69350 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:00.706594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1cb6d-36b4-41b9-9488-c03df3d69350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:00.706873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1cb6d-36b4-41b9-9488-c03df3d69350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:00.707616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1cb6d-36b4-41b9-9488-c03df3d69350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:00.709273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1cb6d-36b4-41b9-9488-c03df3d69350 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:00.721425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1193/2389] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:00.725688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd46600-9b7a-4bf4-92cd-e5bdd78287be admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:00.736416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd46600-9b7a-4bf4-92cd-e5bdd78287be admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:00.741862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7de3a-4fbc-4717-918d-9761d845ef97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:00.742739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7de3a-4fbc-4717-918d-9761d845ef97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:00.743023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7de3a-4fbc-4717-918d-9761d845ef97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:00.743300 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7de3a-4fbc-4717-918d-9761d845ef97 admin admin] PATH_INFO: `/v3/projects/271a1b4b043f48259c0c2b74e9beb27d/users/f99abf81feee46ad842db5aead38d15b/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:00.758441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7de3a-4fbc-4717-918d-9761d845ef97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271a1b4b043f48259c0c2b74e9beb27d, user_id=f99abf81feee46ad842db5aead38d15b, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:00.761076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7de3a-4fbc-4717-918d-9761d845ef97 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:00.873713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1197/2390] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:00 2022] PUT /identity/v3/projects/271a1b4b043f48259c0c2b74e9beb27d/users/f99abf81feee46ad842db5aead38d15b/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:00.878083 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1cb6d-36b4-41b9-9488-c03df3d69350 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:00.897610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1cb6d-36b4-41b9-9488-c03df3d69350 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:00.963352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e72e6-dc4a-43ae-8c5d-2609b8658d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:00.964058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e72e6-dc4a-43ae-8c5d-2609b8658d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:00.964353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e72e6-dc4a-43ae-8c5d-2609b8658d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:00.964600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e72e6-dc4a-43ae-8c5d-2609b8658d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:00.965372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e72e6-dc4a-43ae-8c5d-2609b8658d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:00.974280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e72e6-dc4a-43ae-8c5d-2609b8658d58 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:00.981568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1194/2391] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:00.983261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7de3a-4fbc-4717-918d-9761d845ef97 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:00.994296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7de3a-4fbc-4717-918d-9761d845ef97 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:00.999923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead9413-6efb-4726-b85f-f2cf8575bfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:01.000895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead9413-6efb-4726-b85f-f2cf8575bfc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:01.001206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead9413-6efb-4726-b85f-f2cf8575bfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:01.001504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead9413-6efb-4726-b85f-f2cf8575bfc8 admin admin] PATH_INFO: `/v3/projects/271a1b4b043f48259c0c2b74e9beb27d/users/f99abf81feee46ad842db5aead38d15b/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:01.003706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead9413-6efb-4726-b85f-f2cf8575bfc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271a1b4b043f48259c0c2b74e9beb27d, user_id=f99abf81feee46ad842db5aead38d15b, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:01.006486 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead9413-6efb-4726-b85f-f2cf8575bfc8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:01.018418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1198/2392] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:00 2022] PUT /identity/v3/projects/271a1b4b043f48259c0c2b74e9beb27d/users/f99abf81feee46ad842db5aead38d15b/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:01.029260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02420854-09d2-46af-8589-19d4f7869224 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:01.029551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02420854-09d2-46af-8589-19d4f7869224 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:01.029945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02420854-09d2-46af-8589-19d4f7869224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:01.064629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-02420854-09d2-46af-8589-19d4f7869224 None None] MFA Rules not processed for user `f99abf81feee46ad842db5aead38d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:01.066108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-02420854-09d2-46af-8589-19d4f7869224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:01.134507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:01.144175 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1195/2393] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:01.155597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead9413-6efb-4726-b85f-f2cf8575bfc8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:01.157340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bead9413-6efb-4726-b85f-f2cf8575bfc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:01.217806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead9413-6efb-4726-b85f-f2cf8575bfc8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:01.222342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65da05f-a5b2-4e73-b0c8-707122e4d4e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:01.222999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65da05f-a5b2-4e73-b0c8-707122e4d4e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:01.223308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65da05f-a5b2-4e73-b0c8-707122e4d4e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:01.223742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65da05f-a5b2-4e73-b0c8-707122e4d4e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:01.233695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65da05f-a5b2-4e73-b0c8-707122e4d4e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:01.235204 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65da05f-a5b2-4e73-b0c8-707122e4d4e4 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:01.245193 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1199/2394] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:01.347461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02420854-09d2-46af-8589-19d4f7869224 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:01.357208 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02420854-09d2-46af-8589-19d4f7869224 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:01.360903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16630548-b0ba-4de5-b163-efe043ed4956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:01.361583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16630548-b0ba-4de5-b163-efe043ed4956 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:01.361866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16630548-b0ba-4de5-b163-efe043ed4956 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:01.362143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16630548-b0ba-4de5-b163-efe043ed4956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:01.421793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16630548-b0ba-4de5-b163-efe043ed4956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:01.423250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16630548-b0ba-4de5-b163-efe043ed4956 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:01.433304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1196/2395] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:01.744872 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65da05f-a5b2-4e73-b0c8-707122e4d4e4 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:01.746326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b65da05f-a5b2-4e73-b0c8-707122e4d4e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:01.808120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65da05f-a5b2-4e73-b0c8-707122e4d4e4 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:01.811867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0591f7-4e50-4e60-b331-296f76627fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:01.812549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0591f7-4e50-4e60-b331-296f76627fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:01.812833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0591f7-4e50-4e60-b331-296f76627fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:01.813111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0591f7-4e50-4e60-b331-296f76627fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:01.815266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1d0591f7-4e50-4e60-b331-296f76627fb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:01.882274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:01.885046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0591f7-4e50-4e60-b331-296f76627fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:01.886612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0591f7-4e50-4e60-b331-296f76627fb9 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:01.897093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1200/2396] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:02.744832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16630548-b0ba-4de5-b163-efe043ed4956 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:02.746407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-16630548-b0ba-4de5-b163-efe043ed4956 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:02.828057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16630548-b0ba-4de5-b163-efe043ed4956 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:02.834607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:02.835612 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:02.835940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:02.836230 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] PATH_INFO: `/v3/users/db5d9b69b93a439d849cc3fd2a4ef4e4` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:02.837525 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5d9b69b93a439d849cc3fd2a4ef4e4)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:02.839685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:03.340135 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5d9b69b93a439d849cc3fd2a4ef4e4'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:03.342977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5d9b69b93a439d849cc3fd2a4ef4e4'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:03.378775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5d9b69b93a439d849cc3fd2a4ef4e4'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:03.401697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5d9b69b93a439d849cc3fd2a4ef4e4'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:03.472449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5d9b69b93a439d849cc3fd2a4ef4e4'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:03.483107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1197/2397] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:02 2022] DELETE /identity/v3/users/db5d9b69b93a439d849cc3fd2a4ef4e4 => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:03.484626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1683cc-eb4a-4ecf-9da4-783404b1fff4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:03.485102 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1683cc-eb4a-4ecf-9da4-783404b1fff4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:03.485297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1683cc-eb4a-4ecf-9da4-783404b1fff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:03.502783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7b1683cc-eb4a-4ecf-9da4-783404b1fff4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:03.504011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7b1683cc-eb4a-4ecf-9da4-783404b1fff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:03.587094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1201/2398] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:51:03.596547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:03.598025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:03.612445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:03.659338 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4991f-e83c-428f-a629-870073d20fc1 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:03.664878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15559242-0376-4e1c-aa0b-386417db6d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:03.665716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15559242-0376-4e1c-aa0b-386417db6d9d service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:03.666075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15559242-0376-4e1c-aa0b-386417db6d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:03.666481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15559242-0376-4e1c-aa0b-386417db6d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:03.677854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15559242-0376-4e1c-aa0b-386417db6d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:03.679819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15559242-0376-4e1c-aa0b-386417db6d9d service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:03.690935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1198/2399] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:04.186973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1683cc-eb4a-4ecf-9da4-783404b1fff4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:04.188402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7b1683cc-eb4a-4ecf-9da4-783404b1fff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:04.206926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1683cc-eb4a-4ecf-9da4-783404b1fff4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:04.212480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:04.221560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:04.221852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:04.222135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] PATH_INFO: `/v3/projects/04a48027848d4de7bd9e20c1fae32ae1` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:04.224236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a48027848d4de7bd9e20c1fae32ae1)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:04.225855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:04.347689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a48027848d4de7bd9e20c1fae32ae1', 'request_id': 'req-f2a424c1-4432-45b0-acf4-b284b7b8fb38'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.378499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15559242-0376-4e1c-aa0b-386417db6d9d service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:04.379959 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-15559242-0376-4e1c-aa0b-386417db6d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:04.392800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15559242-0376-4e1c-aa0b-386417db6d9d service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:04.426620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a48027848d4de7bd9e20c1fae32ae1', 'request_id': 'req-f2a424c1-4432-45b0-acf4-b284b7b8fb38'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.440674 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a48027848d4de7bd9e20c1fae32ae1', 'request_id': 'req-f2a424c1-4432-45b0-acf4-b284b7b8fb38'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.456875 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] The token cache is being invalidate because project 04a48027848d4de7bd9e20c1fae32ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:04.457307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.458609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.464059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1202/2400] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:04 2022] DELETE /identity/v3/projects/04a48027848d4de7bd9e20c1fae32ae1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:04.464959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:04.466602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:04.468941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:04.470170 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:04.470597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:04.471562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] PATH_INFO: `/v3/users/f6bf2b373bcd455594426033167bbb97` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:04.472941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6bf2b373bcd455594426033167bbb97)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:04.474965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:04.481666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a424c1-4432-45b0-acf4-b284b7b8fb38 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:04.487239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:04.488703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:04.489171 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:04.489454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] PATH_INFO: `/v3/users/451504e8fb124182b05b8902299ed767` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:04.490673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] RBAC: Authorizing `identity:delete_user(user_id=451504e8fb124182b05b8902299ed767)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:04.492586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:04.700970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6bf2b373bcd455594426033167bbb97'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.701634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6bf2b373bcd455594426033167bbb97'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.716997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451504e8fb124182b05b8902299ed767'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.720984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6bf2b373bcd455594426033167bbb97'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.729436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451504e8fb124182b05b8902299ed767'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.731667 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6bf2b373bcd455594426033167bbb97'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.748850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451504e8fb124182b05b8902299ed767'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.756102 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:04.820485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6bf2b373bcd455594426033167bbb97'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.832476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1199/2401] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:04 2022] DELETE /identity/v3/users/f6bf2b373bcd455594426033167bbb97 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:04.832476 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:04.834661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:04.873455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451504e8fb124182b05b8902299ed767'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.874503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1c1ba08f-c8e2-47d1-abac-833d7a30759d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451504e8fb124182b05b8902299ed767'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:04.889231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1203/2402] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:04 2022] DELETE /identity/v3/users/451504e8fb124182b05b8902299ed767 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:04.891442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4e12f-32f7-4457-9fdc-af06bef0831a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:04.891936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4e12f-32f7-4457-9fdc-af06bef0831a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:04.892425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4e12f-32f7-4457-9fdc-af06bef0831a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:04.922376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-76a4e12f-32f7-4457-9fdc-af06bef0831a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:04.926061 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-76a4e12f-32f7-4457-9fdc-af06bef0831a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:04.938111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:05.025047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f11b9-2057-48df-9ecc-dd57e027f7bb admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:05.072662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1204/2403] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:05.097732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4e12f-32f7-4457-9fdc-af06bef0831a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:05.099192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-76a4e12f-32f7-4457-9fdc-af06bef0831a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:05.115597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:05.117930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:05.118549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:05.119016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] PATH_INFO: `/v3/users/d4782516924347b68f86b0c56ac36d37` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:05.120361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4782516924347b68f86b0c56ac36d37)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:05.123287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:05.191794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4e12f-32f7-4457-9fdc-af06bef0831a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:05.196076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc584e64-1fb7-4cdc-9ac9-915f63d3b80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:05.196737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc584e64-1fb7-4cdc-9ac9-915f63d3b80b service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:05.197023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc584e64-1fb7-4cdc-9ac9-915f63d3b80b service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:05.197450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc584e64-1fb7-4cdc-9ac9-915f63d3b80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:05.209197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc584e64-1fb7-4cdc-9ac9-915f63d3b80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:05.210105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc584e64-1fb7-4cdc-9ac9-915f63d3b80b service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:05.222711 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1205/2404] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:05.327828 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4782516924347b68f86b0c56ac36d37'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:05.328631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4782516924347b68f86b0c56ac36d37'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:05.342903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4782516924347b68f86b0c56ac36d37'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:05.351301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4782516924347b68f86b0c56ac36d37'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:05.420394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4782516924347b68f86b0c56ac36d37'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:05.427791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1200/2405] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:04 2022] DELETE /identity/v3/users/d4782516924347b68f86b0c56ac36d37 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:05.432493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37752453-4a9a-4a6a-8d38-6b65f3387d2e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:05.433105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37752453-4a9a-4a6a-8d38-6b65f3387d2e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:05.433616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37752453-4a9a-4a6a-8d38-6b65f3387d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:05.451161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-37752453-4a9a-4a6a-8d38-6b65f3387d2e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:05.453380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-37752453-4a9a-4a6a-8d38-6b65f3387d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:05.550551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1206/2406] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:05.571429 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:05.573434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:05.648569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62660056-6050-4c6e-a619-802de9671e6e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:05.652496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d797891-5cce-44c1-a28d-3eabde797895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:05.653560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d797891-5cce-44c1-a28d-3eabde797895 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:05.653560 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d797891-5cce-44c1-a28d-3eabde797895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:05.654045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d797891-5cce-44c1-a28d-3eabde797895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:05.666594 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d797891-5cce-44c1-a28d-3eabde797895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:05.669806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d797891-5cce-44c1-a28d-3eabde797895 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:05.679213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1201/2407] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:05.952609 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37752453-4a9a-4a6a-8d38-6b65f3387d2e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:05.956467 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-37752453-4a9a-4a6a-8d38-6b65f3387d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:05.977896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37752453-4a9a-4a6a-8d38-6b65f3387d2e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:05.989465 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:05.990493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:05.990876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:05.991233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] PATH_INFO: `/v3/projects/90aca639f1364d11a57f3ea86f1a90c0` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:05.992591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=90aca639f1364d11a57f3ea86f1a90c0)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:05.996515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:06.087361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d797891-5cce-44c1-a28d-3eabde797895 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:06.095064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3d797891-5cce-44c1-a28d-3eabde797895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:06.105013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:06.113716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d797891-5cce-44c1-a28d-3eabde797895 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:06.122578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:06.163024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90aca639f1364d11a57f3ea86f1a90c0', 'request_id': 'req-a9b8a711-15ce-4d77-9013-440d64de7b3e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.240776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:06.242440 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:06.243367 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:06.244009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] PATH_INFO: `/v3/projects/7f7c1f02b16a479d97d7e55473dbf3a7` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:06.247214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f7c1f02b16a479d97d7e55473dbf3a7)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:06.250010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:06.265454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90aca639f1364d11a57f3ea86f1a90c0', 'request_id': 'req-a9b8a711-15ce-4d77-9013-440d64de7b3e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.286494 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90aca639f1364d11a57f3ea86f1a90c0', 'request_id': 'req-a9b8a711-15ce-4d77-9013-440d64de7b3e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.331230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] The token cache is being invalidate because project 90aca639f1364d11a57f3ea86f1a90c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:06.331523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.335072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a9b8a711-15ce-4d77-9013-440d64de7b3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.339007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1207/2408] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:05 2022] DELETE /identity/v3/projects/90aca639f1364d11a57f3ea86f1a90c0 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:06.345971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c81bcc-6ed4-4507-85fb-e10a6214b876 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:06.346292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c81bcc-6ed4-4507-85fb-e10a6214b876 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:06.346552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c81bcc-6ed4-4507-85fb-e10a6214b876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:06.385612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b2c81bcc-6ed4-4507-85fb-e10a6214b876 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:06.389072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b2c81bcc-6ed4-4507-85fb-e10a6214b876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:06.411833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1208/2409] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:51:06.419636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489b83d-f0f9-4021-b7f8-b91272cc00bf None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:06.419980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489b83d-f0f9-4021-b7f8-b91272cc00bf None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:06.420896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489b83d-f0f9-4021-b7f8-b91272cc00bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:06.429381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f7c1f02b16a479d97d7e55473dbf3a7', 'request_id': 'req-aea50428-fd34-46a7-affa-c2eae7d1094a'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.443744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f7c1f02b16a479d97d7e55473dbf3a7', 'request_id': 'req-aea50428-fd34-46a7-affa-c2eae7d1094a'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.447555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1489b83d-f0f9-4021-b7f8-b91272cc00bf None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:06.448774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1489b83d-f0f9-4021-b7f8-b91272cc00bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:06.450756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f7c1f02b16a479d97d7e55473dbf3a7', 'request_id': 'req-aea50428-fd34-46a7-affa-c2eae7d1094a'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.521870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1209/2410] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:51:06.525659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489b83d-f0f9-4021-b7f8-b91272cc00bf None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:06.537373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489b83d-f0f9-4021-b7f8-b91272cc00bf None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:06.543371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532ed5a-68f9-45b3-8801-36c5c3386095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:06.544312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532ed5a-68f9-45b3-8801-36c5c3386095 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:06.544709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532ed5a-68f9-45b3-8801-36c5c3386095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:06.544982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532ed5a-68f9-45b3-8801-36c5c3386095 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:06.545787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532ed5a-68f9-45b3-8801-36c5c3386095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:06.546099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532ed5a-68f9-45b3-8801-36c5c3386095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:06.547710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532ed5a-68f9-45b3-8801-36c5c3386095 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:06.571724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1210/2411] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:51:06.574619 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532ed5a-68f9-45b3-8801-36c5c3386095 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:06.576642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8532ed5a-68f9-45b3-8801-36c5c3386095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:06.595366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532ed5a-68f9-45b3-8801-36c5c3386095 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:06.601946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] The token cache is being invalidate because project 7f7c1f02b16a479d97d7e55473dbf3a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:06.602233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.603535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.604402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:06.606850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1202/2412] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:06 2022] DELETE /identity/v3/projects/7f7c1f02b16a479d97d7e55473dbf3a7 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:06.609427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:06.610033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:06.610562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] PATH_INFO: `/v3/projects/7a7a7f2803f54dd3b700177a632f4147` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:06.613011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7a7f2803f54dd3b700177a632f4147)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:06.615549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:06.616253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:06.618201 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:06.631271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea50428-fd34-46a7-affa-c2eae7d1094a admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:06.637650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e057fb-ad89-4949-8537-2dd73587c0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:06.638329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e057fb-ad89-4949-8537-2dd73587c0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:06.638797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e057fb-ad89-4949-8537-2dd73587c0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:06.639215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e057fb-ad89-4949-8537-2dd73587c0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:06.640081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e057fb-ad89-4949-8537-2dd73587c0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:06.642169 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e057fb-ad89-4949-8537-2dd73587c0a1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:06.751468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1203/2413] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:06 2022] POST /identity/v3/projects => generated 401 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:06.752412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e057fb-ad89-4949-8537-2dd73587c0a1 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:06.755674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e9e057fb-ad89-4949-8537-2dd73587c0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:06.770166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7a7f2803f54dd3b700177a632f4147', 'request_id': 'req-831be064-de06-4ba4-948c-f31c75131431'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.827018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e057fb-ad89-4949-8537-2dd73587c0a1 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:06.832740 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:06.833493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:06.833780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:06.834055 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] PATH_INFO: `/v3/users/aadf6a8bb9a749cc96c18a9c28c66d7d` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:06.835823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aadf6a8bb9a749cc96c18a9c28c66d7d)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:06.843808 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:06.877893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7a7f2803f54dd3b700177a632f4147', 'request_id': 'req-831be064-de06-4ba4-948c-f31c75131431'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.884206 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7a7f2803f54dd3b700177a632f4147', 'request_id': 'req-831be064-de06-4ba4-948c-f31c75131431'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.896558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] The token cache is being invalidate because project 7a7a7f2803f54dd3b700177a632f4147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:06.896978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.897946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:06.899759 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1211/2414] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:06 2022] DELETE /identity/v3/projects/7a7a7f2803f54dd3b700177a632f4147 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:06.903455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:06.904880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:06.977889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831be064-de06-4ba4-948c-f31c75131431 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:06.985361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df2006-d4af-4473-8173-9a09071fe5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:06.986058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2006-d4af-4473-8173-9a09071fe5eb admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:06.986395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2006-d4af-4473-8173-9a09071fe5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:06.986656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2006-d4af-4473-8173-9a09071fe5eb admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:06.987409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df2006-d4af-4473-8173-9a09071fe5eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:06.989335 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df2006-d4af-4473-8173-9a09071fe5eb admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:07.090365 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1212/2415] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:06 2022] POST /identity/v3/users => generated 301 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:07.091519 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df2006-d4af-4473-8173-9a09071fe5eb admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:07.093732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-96df2006-d4af-4473-8173-9a09071fe5eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:07.102642 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aadf6a8bb9a749cc96c18a9c28c66d7d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:07.103276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aadf6a8bb9a749cc96c18a9c28c66d7d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:07.107602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df2006-d4af-4473-8173-9a09071fe5eb admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:07.112962 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:07.114126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:07.119285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:07.119285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] PATH_INFO: `/v3/users/f99abf81feee46ad842db5aead38d15b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:07.119285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f99abf81feee46ad842db5aead38d15b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:07.119285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:07.125977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aadf6a8bb9a749cc96c18a9c28c66d7d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:07.135902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aadf6a8bb9a749cc96c18a9c28c66d7d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:07.229587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aadf6a8bb9a749cc96c18a9c28c66d7d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:07.238589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1204/2416] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:06 2022] DELETE /identity/v3/users/aadf6a8bb9a749cc96c18a9c28c66d7d => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:07.239489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:07.241074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:07.335633 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5deaa3-9d68-4dab-913c-fcc829054652 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:07.342402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1ab27-1172-453d-afa3-58539eb6a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:07.343195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1ab27-1172-453d-afa3-58539eb6a92c admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:07.344002 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1ab27-1172-453d-afa3-58539eb6a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:07.347102 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1ab27-1172-453d-afa3-58539eb6a92c admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:07.368056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1ab27-1172-453d-afa3-58539eb6a92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:07.373328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1ab27-1172-453d-afa3-58539eb6a92c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:07.381679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f99abf81feee46ad842db5aead38d15b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:07.389996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1205/2417] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:07.393638 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f99abf81feee46ad842db5aead38d15b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:07.394547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1ab27-1172-453d-afa3-58539eb6a92c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:07.405073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1ab27-1172-453d-afa3-58539eb6a92c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:07.409516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f99abf81feee46ad842db5aead38d15b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:07.410508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c19f82-9963-4de1-8464-1a2173aee33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:07.411183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c19f82-9963-4de1-8464-1a2173aee33b admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:07.411520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c19f82-9963-4de1-8464-1a2173aee33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:07.411733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c19f82-9963-4de1-8464-1a2173aee33b admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:07.412482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c19f82-9963-4de1-8464-1a2173aee33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:07.414121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c19f82-9963-4de1-8464-1a2173aee33b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:07.424444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1206/2418] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:51:07.428619 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c19f82-9963-4de1-8464-1a2173aee33b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:07.438490 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c19f82-9963-4de1-8464-1a2173aee33b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:07.443703 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dd37e-165c-443b-93a2-dbd778f87bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:07.444578 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dd37e-165c-443b-93a2-dbd778f87bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:07.450273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dd37e-165c-443b-93a2-dbd778f87bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:07.450273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dd37e-165c-443b-93a2-dbd778f87bcd admin admin] PATH_INFO: `/v3/projects/3c4213e0154f429fa1b268d98618ce1b/users/4e8b405029c34e92971172e2081d3915/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:07.460352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dd37e-165c-443b-93a2-dbd778f87bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4213e0154f429fa1b268d98618ce1b, user_id=4e8b405029c34e92971172e2081d3915, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:07.463000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dd37e-165c-443b-93a2-dbd778f87bcd admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:07.527517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f99abf81feee46ad842db5aead38d15b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:07.528420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-81aba816-4f80-4d55-b733-75f0de6463d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f99abf81feee46ad842db5aead38d15b'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:07.536160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1213/2419] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:07 2022] DELETE /identity/v3/users/f99abf81feee46ad842db5aead38d15b => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:07.609250 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1207/2420] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:07 2022] PUT /identity/v3/projects/3c4213e0154f429fa1b268d98618ce1b/users/4e8b405029c34e92971172e2081d3915/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:07.625317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03003d-9713-4162-b4fc-ed30e631dce1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:07.625899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03003d-9713-4162-b4fc-ed30e631dce1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:07.626196 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03003d-9713-4162-b4fc-ed30e631dce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:07.671837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0c03003d-9713-4162-b4fc-ed30e631dce1 None None] MFA Rules not processed for user `4e8b405029c34e92971172e2081d3915`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:07.675321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0c03003d-9713-4162-b4fc-ed30e631dce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:07.754224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1214/2421] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:07.758946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dd37e-165c-443b-93a2-dbd778f87bcd admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:07.760853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-695dd37e-165c-443b-93a2-dbd778f87bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:07.858713 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dd37e-165c-443b-93a2-dbd778f87bcd admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:07.864093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45236a50-b83f-40ae-aec7-e22d8a46e898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:07.864768 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45236a50-b83f-40ae-aec7-e22d8a46e898 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:07.865054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45236a50-b83f-40ae-aec7-e22d8a46e898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:07.865359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45236a50-b83f-40ae-aec7-e22d8a46e898 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:07.866062 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45236a50-b83f-40ae-aec7-e22d8a46e898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:07.868197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45236a50-b83f-40ae-aec7-e22d8a46e898 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:07.928641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03003d-9713-4162-b4fc-ed30e631dce1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:07.931578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0c03003d-9713-4162-b4fc-ed30e631dce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:07.950242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03003d-9713-4162-b4fc-ed30e631dce1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:07.957874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:07.959575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:07.960024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:07.960409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] PATH_INFO: `/v3/projects/75f64634e4a84fbfb4e025508e357e8c` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:07.961922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f64634e4a84fbfb4e025508e357e8c)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:07.967340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:07.973467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1208/2422] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:07 2022] POST /identity/v3/projects => generated 399 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:07.980871 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45236a50-b83f-40ae-aec7-e22d8a46e898 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:08.078589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45236a50-b83f-40ae-aec7-e22d8a46e898 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:08.088761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafb043-cb8d-4d58-b2ed-83050339b081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:08.089509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafb043-cb8d-4d58-b2ed-83050339b081 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.089781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafb043-cb8d-4d58-b2ed-83050339b081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.090056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafb043-cb8d-4d58-b2ed-83050339b081 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.090760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafb043-cb8d-4d58-b2ed-83050339b081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:08.093255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafb043-cb8d-4d58-b2ed-83050339b081 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:08.111730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f64634e4a84fbfb4e025508e357e8c', 'request_id': 'req-7754aa06-ec77-46dc-a71d-88916f65fc40'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:08.123643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:08.215398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:08.265992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f64634e4a84fbfb4e025508e357e8c', 'request_id': 'req-7754aa06-ec77-46dc-a71d-88916f65fc40'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:08.274464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1209/2423] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:07 2022] POST /identity/v3/users => generated 306 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:08.278596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafb043-cb8d-4d58-b2ed-83050339b081 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:08.286839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f64634e4a84fbfb4e025508e357e8c', 'request_id': 'req-7754aa06-ec77-46dc-a71d-88916f65fc40'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:08.289034 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0cafb043-cb8d-4d58-b2ed-83050339b081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:08.306032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] The token cache is being invalidate because project 75f64634e4a84fbfb4e025508e357e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:08.306630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:08.307491 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:08.310295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafb043-cb8d-4d58-b2ed-83050339b081 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:08.311452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1215/2424] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:07 2022] DELETE /identity/v3/projects/75f64634e4a84fbfb4e025508e357e8c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:08.316317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:08.317981 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:08.402945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:08.403738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.404073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.404351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] PATH_INFO: `/v3/projects/271a1b4b043f48259c0c2b74e9beb27d` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.405650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=271a1b4b043f48259c0c2b74e9beb27d)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:08.407732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:08.419627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754aa06-ec77-46dc-a71d-88916f65fc40 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:08.425522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c62615-5fc6-4851-8083-a3197c60a5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:08.430805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c62615-5fc6-4851-8083-a3197c60a5ed admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.430805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c62615-5fc6-4851-8083-a3197c60a5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.430805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c62615-5fc6-4851-8083-a3197c60a5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.430805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c62615-5fc6-4851-8083-a3197c60a5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:08.430805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c62615-5fc6-4851-8083-a3197c60a5ed admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:08.440967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1216/2425] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:08.442495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a320d-2427-43c2-9c49-c638f14e4daf None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.442786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a320d-2427-43c2-9c49-c638f14e4daf None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.443157 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a320d-2427-43c2-9c49-c638f14e4daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.477229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-088a320d-2427-43c2-9c49-c638f14e4daf None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:08.478630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-088a320d-2427-43c2-9c49-c638f14e4daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:08.495095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1217/2426] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:08.504175 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a320d-2427-43c2-9c49-c638f14e4daf None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:08.514476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a320d-2427-43c2-9c49-c638f14e4daf None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:08.519622 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bbd3a-08e0-44fe-bb77-d8771dd76b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:08.520541 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bbd3a-08e0-44fe-bb77-d8771dd76b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.520862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bbd3a-08e0-44fe-bb77-d8771dd76b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.521089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bbd3a-08e0-44fe-bb77-d8771dd76b8d admin admin] PATH_INFO: `/v3/projects/d5691d4e7b454065be1b217b5ffb001d/users/c93f93bd98a8411e8a94e69bef2d5ac1/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.537484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bbd3a-08e0-44fe-bb77-d8771dd76b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5691d4e7b454065be1b217b5ffb001d, user_id=c93f93bd98a8411e8a94e69bef2d5ac1, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:08.540163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bbd3a-08e0-44fe-bb77-d8771dd76b8d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:08.633651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1218/2427] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:08 2022] PUT /identity/v3/projects/d5691d4e7b454065be1b217b5ffb001d/users/c93f93bd98a8411e8a94e69bef2d5ac1/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:08.636146 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c64b-63e4-4728-a221-97901fafeef3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.636629 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c64b-63e4-4728-a221-97901fafeef3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.637077 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8c64b-63e4-4728-a221-97901fafeef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.648093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271a1b4b043f48259c0c2b74e9beb27d', 'request_id': 'req-62788ffb-712b-4b2e-a193-25d631426862'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:08.656796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-13d8c64b-63e4-4728-a221-97901fafeef3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:08.657977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-13d8c64b-63e4-4728-a221-97901fafeef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:08.658930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271a1b4b043f48259c0c2b74e9beb27d', 'request_id': 'req-62788ffb-712b-4b2e-a193-25d631426862'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:08.666933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271a1b4b043f48259c0c2b74e9beb27d', 'request_id': 'req-62788ffb-712b-4b2e-a193-25d631426862'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:08.725387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1219/2428] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:08.726427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8c64b-63e4-4728-a221-97901fafeef3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:08.727806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-13d8c64b-63e4-4728-a221-97901fafeef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:08.754785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] The token cache is being invalidate because project 271a1b4b043f48259c0c2b74e9beb27d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:08.755215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:08.755907 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:08.760843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1210/2429] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:08 2022] DELETE /identity/v3/projects/271a1b4b043f48259c0c2b74e9beb27d => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:08.760843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:08.760843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:08.773859 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62788ffb-712b-4b2e-a193-25d631426862 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:08.778480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd40756-992b-4b4c-88b0-ed949ddf475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:08.780316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd40756-992b-4b4c-88b0-ed949ddf475c admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.780316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd40756-992b-4b4c-88b0-ed949ddf475c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.780316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd40756-992b-4b4c-88b0-ed949ddf475c admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.780596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd40756-992b-4b4c-88b0-ed949ddf475c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:08.782064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd40756-992b-4b4c-88b0-ed949ddf475c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:08.786016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8c64b-63e4-4728-a221-97901fafeef3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:08.792878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1211/2430] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:08.793759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd40756-992b-4b4c-88b0-ed949ddf475c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:08.795411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ddd40756-992b-4b4c-88b0-ed949ddf475c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:08.798418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001e3f5-165b-4d01-9ca7-1c9ab39507b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:08.799096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001e3f5-165b-4d01-9ca7-1c9ab39507b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.799384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001e3f5-165b-4d01-9ca7-1c9ab39507b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.799658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001e3f5-165b-4d01-9ca7-1c9ab39507b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.801586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e001e3f5-165b-4d01-9ca7-1c9ab39507b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:08.810188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd40756-992b-4b4c-88b0-ed949ddf475c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:08.818938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001e3f5-165b-4d01-9ca7-1c9ab39507b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:08.819461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ae6cb-fc0d-42e7-a726-b434d1e66189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:08.820497 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001e3f5-165b-4d01-9ca7-1c9ab39507b5 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:08.820766 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ae6cb-fc0d-42e7-a726-b434d1e66189 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.821236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ae6cb-fc0d-42e7-a726-b434d1e66189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.821780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ae6cb-fc0d-42e7-a726-b434d1e66189 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.822683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ae6cb-fc0d-42e7-a726-b434d1e66189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:08.823298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ae6cb-fc0d-42e7-a726-b434d1e66189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:08.825411 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ae6cb-fc0d-42e7-a726-b434d1e66189 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:08.831479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1220/2431] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:08.833014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78254-1561-4805-ac1a-66100290838e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.833014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78254-1561-4805-ac1a-66100290838e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.833667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78254-1561-4805-ac1a-66100290838e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.851152 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1212/2432] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:51:08.852629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ae6cb-fc0d-42e7-a726-b434d1e66189 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:08.859596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9eb78254-1561-4805-ac1a-66100290838e None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:08.860871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9eb78254-1561-4805-ac1a-66100290838e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:08.865446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ae6cb-fc0d-42e7-a726-b434d1e66189 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:08.871957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c94b9-890c-498d-bce3-559159b05fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:08.872200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c94b9-890c-498d-bce3-559159b05fab admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.872550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c94b9-890c-498d-bce3-559159b05fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.872733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c94b9-890c-498d-bce3-559159b05fab admin admin] PATH_INFO: `/v3/projects/d5691d4e7b454065be1b217b5ffb001d/users/c93f93bd98a8411e8a94e69bef2d5ac1/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.874967 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c94b9-890c-498d-bce3-559159b05fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5691d4e7b454065be1b217b5ffb001d, user_id=c93f93bd98a8411e8a94e69bef2d5ac1, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:08.878269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1221/2433] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:08.878422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c94b9-890c-498d-bce3-559159b05fab admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:08.886784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb78254-1561-4805-ac1a-66100290838e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:08.891685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1213/2434] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:08 2022] PUT /identity/v3/projects/d5691d4e7b454065be1b217b5ffb001d/users/c93f93bd98a8411e8a94e69bef2d5ac1/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:08.892243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1574c-930e-4c04-aa07-3392aa48f41c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:08.892408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1574c-930e-4c04-aa07-3392aa48f41c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:08.893309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1574c-930e-4c04-aa07-3392aa48f41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:08.916352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-05a1574c-930e-4c04-aa07-3392aa48f41c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:08.918295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-05a1574c-930e-4c04-aa07-3392aa48f41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:08.925497 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb78254-1561-4805-ac1a-66100290838e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.021943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31563e31-59b0-4202-8b26-23ce7c5eeb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.022887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31563e31-59b0-4202-8b26-23ce7c5eeb8d admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.024285 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31563e31-59b0-4202-8b26-23ce7c5eeb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.024892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31563e31-59b0-4202-8b26-23ce7c5eeb8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.025603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31563e31-59b0-4202-8b26-23ce7c5eeb8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.026844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1214/2435] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:09.027603 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31563e31-59b0-4202-8b26-23ce7c5eeb8d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.032122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac1db1-2231-43e7-b93f-392fd3f70a0d None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.032567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac1db1-2231-43e7-b93f-392fd3f70a0d None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.032743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac1db1-2231-43e7-b93f-392fd3f70a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.058789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-21ac1db1-2231-43e7-b93f-392fd3f70a0d None None] MFA Rules not processed for user `c93f93bd98a8411e8a94e69bef2d5ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:09.060070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-21ac1db1-2231-43e7-b93f-392fd3f70a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:09.132649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1215/2436] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:09 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:09.132649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac1db1-2231-43e7-b93f-392fd3f70a0d None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.135108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1222/2437] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:08 2022] POST /identity/v3/projects => generated 365 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:09.138876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31563e31-59b0-4202-8b26-23ce7c5eeb8d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.150751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac1db1-2231-43e7-b93f-392fd3f70a0d None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.218704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b3710-9305-496a-8157-f07b1896332b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.219507 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b3710-9305-496a-8157-f07b1896332b admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.219795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b3710-9305-496a-8157-f07b1896332b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.220070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b3710-9305-496a-8157-f07b1896332b admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.220932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b3710-9305-496a-8157-f07b1896332b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:09.221256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b3710-9305-496a-8157-f07b1896332b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.222339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31563e31-59b0-4202-8b26-23ce7c5eeb8d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.223296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b3710-9305-496a-8157-f07b1896332b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.227833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd425379-8d54-4c7e-9240-64ed522d0e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.228620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd425379-8d54-4c7e-9240-64ed522d0e3a admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.229010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd425379-8d54-4c7e-9240-64ed522d0e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.229450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd425379-8d54-4c7e-9240-64ed522d0e3a admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.230258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd425379-8d54-4c7e-9240-64ed522d0e3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.232276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd425379-8d54-4c7e-9240-64ed522d0e3a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.241031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1216/2438] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:51:09.241031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b3710-9305-496a-8157-f07b1896332b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.241341 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-430b3710-9305-496a-8157-f07b1896332b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:09.302903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b3710-9305-496a-8157-f07b1896332b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.309484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7ffa7-3e47-4e67-9473-21aabcd90ff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.310141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7ffa7-3e47-4e67-9473-21aabcd90ff4 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.310472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7ffa7-3e47-4e67-9473-21aabcd90ff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.310754 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7ffa7-3e47-4e67-9473-21aabcd90ff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.331952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1223/2439] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:09 2022] POST /identity/v3/users => generated 283 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:09.331952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd425379-8d54-4c7e-9240-64ed522d0e3a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.344438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd425379-8d54-4c7e-9240-64ed522d0e3a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.350926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f5512-fcfd-4da2-bd63-96167556a44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.351702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f5512-fcfd-4da2-bd63-96167556a44c admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.352215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f5512-fcfd-4da2-bd63-96167556a44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.352568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f5512-fcfd-4da2-bd63-96167556a44c admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.353445 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f5512-fcfd-4da2-bd63-96167556a44c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.355425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f5512-fcfd-4da2-bd63-96167556a44c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.388562 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7ffa7-3e47-4e67-9473-21aabcd90ff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.390081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7ffa7-3e47-4e67-9473-21aabcd90ff4 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.400617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1217/2440] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:09 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:09.401501 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7ffa7-3e47-4e67-9473-21aabcd90ff4 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.411807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7ffa7-3e47-4e67-9473-21aabcd90ff4 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.417459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea251b9e-3f17-43af-8693-9fa0f4b4fb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.418038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251b9e-3f17-43af-8693-9fa0f4b4fb7f admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.418372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251b9e-3f17-43af-8693-9fa0f4b4fb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.418492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251b9e-3f17-43af-8693-9fa0f4b4fb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.419330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea251b9e-3f17-43af-8693-9fa0f4b4fb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.420977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea251b9e-3f17-43af-8693-9fa0f4b4fb7f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.434616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1218/2441] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:09.437193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea251b9e-3f17-43af-8693-9fa0f4b4fb7f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.448440 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea251b9e-3f17-43af-8693-9fa0f4b4fb7f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.453286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab7a68-0b86-462f-95f2-e10bafcde2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.453888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab7a68-0b86-462f-95f2-e10bafcde2dd admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.454715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab7a68-0b86-462f-95f2-e10bafcde2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.455132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab7a68-0b86-462f-95f2-e10bafcde2dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.455800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab7a68-0b86-462f-95f2-e10bafcde2dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.457412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab7a68-0b86-462f-95f2-e10bafcde2dd admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.467272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1219/2442] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:51:09.471597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab7a68-0b86-462f-95f2-e10bafcde2dd admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.481936 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab7a68-0b86-462f-95f2-e10bafcde2dd admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.488035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1224/2443] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:09 2022] POST /identity/v3/projects => generated 389 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:09.492077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a123d-05c1-4d30-bf82-6956b42cb6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.492077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a123d-05c1-4d30-bf82-6956b42cb6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.492077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a123d-05c1-4d30-bf82-6956b42cb6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.492077 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a123d-05c1-4d30-bf82-6956b42cb6dc admin admin] PATH_INFO: `/v3/projects/57f0359b89e040719cfcf0cba004adae/users/3fc4667a945946f1a0d9876843463cf7/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.493431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f5512-fcfd-4da2-bd63-96167556a44c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.523156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a123d-05c1-4d30-bf82-6956b42cb6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f0359b89e040719cfcf0cba004adae, user_id=3fc4667a945946f1a0d9876843463cf7, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.529535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a123d-05c1-4d30-bf82-6956b42cb6dc admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.564205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f5512-fcfd-4da2-bd63-96167556a44c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.569034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa131ab-d00d-4f74-b482-728d8f38c22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.569732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa131ab-d00d-4f74-b482-728d8f38c22c admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.570009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa131ab-d00d-4f74-b482-728d8f38c22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.570287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa131ab-d00d-4f74-b482-728d8f38c22c admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.570970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa131ab-d00d-4f74-b482-728d8f38c22c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.572882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa131ab-d00d-4f74-b482-728d8f38c22c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.643335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1220/2444] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:09 2022] PUT /identity/v3/projects/57f0359b89e040719cfcf0cba004adae/users/3fc4667a945946f1a0d9876843463cf7/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:09.650940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a123d-05c1-4d30-bf82-6956b42cb6dc admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.704593 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1225/2445] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:09 2022] POST /identity/v3/users => generated 295 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:09.707364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dd7a1-56c2-4004-9991-5dec66a5d126 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.707967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dd7a1-56c2-4004-9991-5dec66a5d126 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.708302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dd7a1-56c2-4004-9991-5dec66a5d126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.710831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a123d-05c1-4d30-bf82-6956b42cb6dc admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.720117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b8452-04c4-400d-9c09-df29b13d4e13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.721359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b8452-04c4-400d-9c09-df29b13d4e13 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.721644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b8452-04c4-400d-9c09-df29b13d4e13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.721921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b8452-04c4-400d-9c09-df29b13d4e13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.723849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-904b8452-04c4-400d-9c09-df29b13d4e13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:09.741262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c71dd7a1-56c2-4004-9991-5dec66a5d126 None None] MFA Rules not processed for user `3fc4667a945946f1a0d9876843463cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:09.742932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c71dd7a1-56c2-4004-9991-5dec66a5d126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:09.806364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b8452-04c4-400d-9c09-df29b13d4e13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.808536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b8452-04c4-400d-9c09-df29b13d4e13 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.813616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1226/2446] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:09.815318 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dd7a1-56c2-4004-9991-5dec66a5d126 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.819696 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1221/2447] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:09.820523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b8452-04c4-400d-9c09-df29b13d4e13 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.892632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b8452-04c4-400d-9c09-df29b13d4e13 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.892857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dd7a1-56c2-4004-9991-5dec66a5d126 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.898037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70d758-66f1-41a0-90b0-8a6a7c0e7717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.898785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70d758-66f1-41a0-90b0-8a6a7c0e7717 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.899069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70d758-66f1-41a0-90b0-8a6a7c0e7717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.899348 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70d758-66f1-41a0-90b0-8a6a7c0e7717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.900185 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70d758-66f1-41a0-90b0-8a6a7c0e7717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.901863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70d758-66f1-41a0-90b0-8a6a7c0e7717 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.904663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fc5f2-765f-48d5-b88d-d83b854a58c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.905441 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fc5f2-765f-48d5-b88d-d83b854a58c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.905728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fc5f2-765f-48d5-b88d-d83b854a58c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.906001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fc5f2-765f-48d5-b88d-d83b854a58c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.906781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fc5f2-765f-48d5-b88d-d83b854a58c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.909387 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fc5f2-765f-48d5-b88d-d83b854a58c1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.917325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1227/2448] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:09.918919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70d758-66f1-41a0-90b0-8a6a7c0e7717 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.926485 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:09.932614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70d758-66f1-41a0-90b0-8a6a7c0e7717 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.941950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a293d-c1c8-46af-a72e-cb6143f2c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.945630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a293d-c1c8-46af-a72e-cb6143f2c12a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.946042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a293d-c1c8-46af-a72e-cb6143f2c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.946338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a293d-c1c8-46af-a72e-cb6143f2c12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:09.947286 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a293d-c1c8-46af-a72e-cb6143f2c12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:09.952844 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a293d-c1c8-46af-a72e-cb6143f2c12a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:09.969357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1228/2449] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:09.974046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a293d-c1c8-46af-a72e-cb6143f2c12a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:09.987856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a293d-c1c8-46af-a72e-cb6143f2c12a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:09.990431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:09.994388 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe17c4-2ffc-4f3f-9c9e-6ba7e1ef2ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:09.995346 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe17c4-2ffc-4f3f-9c9e-6ba7e1ef2ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:09.995661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe17c4-2ffc-4f3f-9c9e-6ba7e1ef2ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:09.995941 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe17c4-2ffc-4f3f-9c9e-6ba7e1ef2ced admin admin] PATH_INFO: `/v3/projects/1da1a758146842ba903abae94ec129c1/users/355fc0d588124aa5b775b05f8b34bfc4/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.016851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1222/2450] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:09 2022] POST /identity/v3/projects => generated 365 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:10.022814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fc5f2-765f-48d5-b88d-d83b854a58c1 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:10.036656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe17c4-2ffc-4f3f-9c9e-6ba7e1ef2ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da1a758146842ba903abae94ec129c1, user_id=355fc0d588124aa5b775b05f8b34bfc4, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:10.039558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe17c4-2ffc-4f3f-9c9e-6ba7e1ef2ced admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:10.102422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fc5f2-765f-48d5-b88d-d83b854a58c1 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:10.115436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76ce00-04b0-4a0e-8d21-186236357221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:10.116281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76ce00-04b0-4a0e-8d21-186236357221 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:10.116564 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76ce00-04b0-4a0e-8d21-186236357221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:10.116842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76ce00-04b0-4a0e-8d21-186236357221 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.117582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76ce00-04b0-4a0e-8d21-186236357221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:10.119513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76ce00-04b0-4a0e-8d21-186236357221 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:10.141652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1229/2451] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:09 2022] PUT /identity/v3/projects/1da1a758146842ba903abae94ec129c1/users/355fc0d588124aa5b775b05f8b34bfc4/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:10.155680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615913f-75a9-4bb4-8f4a-8e31f04df370 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:10.156303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615913f-75a9-4bb4-8f4a-8e31f04df370 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:10.156898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615913f-75a9-4bb4-8f4a-8e31f04df370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.197671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0615913f-75a9-4bb4-8f4a-8e31f04df370 None None] MFA Rules not processed for user `355fc0d588124aa5b775b05f8b34bfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:10.201528 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0615913f-75a9-4bb4-8f4a-8e31f04df370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:10.224915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1223/2452] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:10 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:10.226089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76ce00-04b0-4a0e-8d21-186236357221 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:10.236437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76ce00-04b0-4a0e-8d21-186236357221 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:10.283930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1230/2453] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:10.285049 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615913f-75a9-4bb4-8f4a-8e31f04df370 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:10.336582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096867c7-cd9f-44ad-a986-0d063ebfff9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:10.336582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096867c7-cd9f-44ad-a986-0d063ebfff9f admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:10.336582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096867c7-cd9f-44ad-a986-0d063ebfff9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:10.336582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096867c7-cd9f-44ad-a986-0d063ebfff9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.338475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096867c7-cd9f-44ad-a986-0d063ebfff9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:10.342149 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096867c7-cd9f-44ad-a986-0d063ebfff9f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:10.352066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1224/2454] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:10.356349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096867c7-cd9f-44ad-a986-0d063ebfff9f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:10.369031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096867c7-cd9f-44ad-a986-0d063ebfff9f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:10.387228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1072001-21ec-49e4-b7f6-608f1b607964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:10.388305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1072001-21ec-49e4-b7f6-608f1b607964 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:10.388741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1072001-21ec-49e4-b7f6-608f1b607964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:10.389116 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1072001-21ec-49e4-b7f6-608f1b607964 admin admin] PATH_INFO: `/v3/projects/81f509c9dc3e45aca270e298fdfbcf3f/users/bef3933fda844a569102eae46a3038cc/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.390776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615913f-75a9-4bb4-8f4a-8e31f04df370 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:10.403478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59197e3c-3fe1-4367-8c24-4ee814def001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:10.405906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59197e3c-3fe1-4367-8c24-4ee814def001 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:10.405906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59197e3c-3fe1-4367-8c24-4ee814def001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:10.405906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59197e3c-3fe1-4367-8c24-4ee814def001 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.405906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59197e3c-3fe1-4367-8c24-4ee814def001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:10.407325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59197e3c-3fe1-4367-8c24-4ee814def001 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:10.409525 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1072001-21ec-49e4-b7f6-608f1b607964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f509c9dc3e45aca270e298fdfbcf3f, user_id=bef3933fda844a569102eae46a3038cc, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:10.413243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1072001-21ec-49e4-b7f6-608f1b607964 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:10.558431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1225/2455] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:10 2022] PUT /identity/v3/projects/81f509c9dc3e45aca270e298fdfbcf3f/users/bef3933fda844a569102eae46a3038cc/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:10.562276 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1072001-21ec-49e4-b7f6-608f1b607964 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:10.616998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1231/2456] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:10 2022] POST /identity/v3/projects => generated 389 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:10.620373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59197e3c-3fe1-4367-8c24-4ee814def001 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:10.642353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1072001-21ec-49e4-b7f6-608f1b607964 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:10.731574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59197e3c-3fe1-4367-8c24-4ee814def001 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:10.742280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8751e-bb1a-485a-a0a9-adb1755aee25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:10.743168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8751e-bb1a-485a-a0a9-adb1755aee25 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:10.743553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8751e-bb1a-485a-a0a9-adb1755aee25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:10.743932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8751e-bb1a-485a-a0a9-adb1755aee25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.744811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8751e-bb1a-485a-a0a9-adb1755aee25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:10.746602 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8751e-bb1a-485a-a0a9-adb1755aee25 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:10.747467 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd55b6-deca-449f-8a2e-464351aa07d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:10.748161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd55b6-deca-449f-8a2e-464351aa07d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:10.748443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd55b6-deca-449f-8a2e-464351aa07d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:10.748713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd55b6-deca-449f-8a2e-464351aa07d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.749471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd55b6-deca-449f-8a2e-464351aa07d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:10.751407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd55b6-deca-449f-8a2e-464351aa07d8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:10.759932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1226/2457] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:51:10.764380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8751e-bb1a-485a-a0a9-adb1755aee25 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:10.776703 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8751e-bb1a-485a-a0a9-adb1755aee25 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:10.787520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e1f87-2c5a-4697-9975-97ab318b8ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:10.787520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e1f87-2c5a-4697-9975-97ab318b8ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:10.787520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e1f87-2c5a-4697-9975-97ab318b8ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:10.787520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e1f87-2c5a-4697-9975-97ab318b8ed2 admin admin] PATH_INFO: `/v3/projects/81f509c9dc3e45aca270e298fdfbcf3f/users/bef3933fda844a569102eae46a3038cc/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.789577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e1f87-2c5a-4697-9975-97ab318b8ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f509c9dc3e45aca270e298fdfbcf3f, user_id=bef3933fda844a569102eae46a3038cc, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:10.793365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e1f87-2c5a-4697-9975-97ab318b8ed2 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:10.806724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1227/2458] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:10 2022] PUT /identity/v3/projects/81f509c9dc3e45aca270e298fdfbcf3f/users/bef3933fda844a569102eae46a3038cc/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:10.818086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1e198-911d-4ec0-9453-714552b1fd50 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:10.818396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1e198-911d-4ec0-9453-714552b1fd50 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:10.819291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1e198-911d-4ec0-9453-714552b1fd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.845795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-f2e1e198-911d-4ec0-9453-714552b1fd50 None None] MFA Rules not processed for user `bef3933fda844a569102eae46a3038cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:10.847018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f2e1e198-911d-4ec0-9453-714552b1fd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:10.865006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1232/2459] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:10 2022] POST /identity/v3/users => generated 301 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:10.871626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd55b6-deca-449f-8a2e-464351aa07d8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:10.886431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd55b6-deca-449f-8a2e-464351aa07d8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:10.971359 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175aa14-a3f6-437c-8daf-76e5e63e240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:10.972263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175aa14-a3f6-437c-8daf-76e5e63e240b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:10.972743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175aa14-a3f6-437c-8daf-76e5e63e240b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:10.973091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175aa14-a3f6-437c-8daf-76e5e63e240b admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:10.973947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175aa14-a3f6-437c-8daf-76e5e63e240b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:10.980871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175aa14-a3f6-437c-8daf-76e5e63e240b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:10.989380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1233/2460] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:10.993994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175aa14-a3f6-437c-8daf-76e5e63e240b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:11.025126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175aa14-a3f6-437c-8daf-76e5e63e240b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:11.041759 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c6bac-636c-4942-9d73-48bcfef3fcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:11.043880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c6bac-636c-4942-9d73-48bcfef3fcb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:11.044670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c6bac-636c-4942-9d73-48bcfef3fcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:11.045423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c6bac-636c-4942-9d73-48bcfef3fcb3 admin admin] PATH_INFO: `/v3/projects/326f2d3010294e2c9487409a298e464c/users/eb90ad7fea3b49c2b91639a7e8476b41/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:11.062785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1228/2461] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:51:11.081113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1e198-911d-4ec0-9453-714552b1fd50 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:11.083936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c6bac-636c-4942-9d73-48bcfef3fcb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=326f2d3010294e2c9487409a298e464c, user_id=eb90ad7fea3b49c2b91639a7e8476b41, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:11.086690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c6bac-636c-4942-9d73-48bcfef3fcb3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:11.165221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1e198-911d-4ec0-9453-714552b1fd50 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:11.168993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1c887-2b79-41ed-8099-90561eedc7b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:11.169790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1c887-2b79-41ed-8099-90561eedc7b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:11.170074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1c887-2b79-41ed-8099-90561eedc7b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:11.170358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1c887-2b79-41ed-8099-90561eedc7b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:11.180184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1c887-2b79-41ed-8099-90561eedc7b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:11.181733 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1c887-2b79-41ed-8099-90561eedc7b3 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:11.192023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1229/2462] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:11.198813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1234/2463] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:10 2022] PUT /identity/v3/projects/326f2d3010294e2c9487409a298e464c/users/eb90ad7fea3b49c2b91639a7e8476b41/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:11.202693 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1c887-2b79-41ed-8099-90561eedc7b3 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:11.218558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1c887-2b79-41ed-8099-90561eedc7b3 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:11.273472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:11.279758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590b94c-aea2-405f-8a4f-e872d0436489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:11.280544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590b94c-aea2-405f-8a4f-e872d0436489 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:11.280823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590b94c-aea2-405f-8a4f-e872d0436489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:11.281133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590b94c-aea2-405f-8a4f-e872d0436489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:11.281852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590b94c-aea2-405f-8a4f-e872d0436489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:11.283442 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590b94c-aea2-405f-8a4f-e872d0436489 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:11.295111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1230/2464] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:11.299452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c6bac-636c-4942-9d73-48bcfef3fcb3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:11.310291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c6bac-636c-4942-9d73-48bcfef3fcb3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:11.315498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b53d9b-7a14-4b28-8f7b-e4e92f927ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:11.316647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b53d9b-7a14-4b28-8f7b-e4e92f927ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:11.318627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b53d9b-7a14-4b28-8f7b-e4e92f927ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:11.318627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b53d9b-7a14-4b28-8f7b-e4e92f927ef3 admin admin] PATH_INFO: `/v3/projects/326f2d3010294e2c9487409a298e464c/users/eb90ad7fea3b49c2b91639a7e8476b41/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:11.319811 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b53d9b-7a14-4b28-8f7b-e4e92f927ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=326f2d3010294e2c9487409a298e464c, user_id=eb90ad7fea3b49c2b91639a7e8476b41, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:11.322631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b53d9b-7a14-4b28-8f7b-e4e92f927ef3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:11.336929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1235/2465] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:11 2022] PUT /identity/v3/projects/326f2d3010294e2c9487409a298e464c/users/eb90ad7fea3b49c2b91639a7e8476b41/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:11.338997 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:11.347014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96637ef6-6f62-4994-aeb5-3d8dd6ce39a4 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:11.347304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96637ef6-6f62-4994-aeb5-3d8dd6ce39a4 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:11.347686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96637ef6-6f62-4994-aeb5-3d8dd6ce39a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:11.379281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-96637ef6-6f62-4994-aeb5-3d8dd6ce39a4 None None] MFA Rules not processed for user `eb90ad7fea3b49c2b91639a7e8476b41`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:11.380511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-96637ef6-6f62-4994-aeb5-3d8dd6ce39a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:11.453824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b53d9b-7a14-4b28-8f7b-e4e92f927ef3 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:11.471680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1231/2466] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:11.485108 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96637ef6-6f62-4994-aeb5-3d8dd6ce39a4 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:11.487221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-96637ef6-6f62-4994-aeb5-3d8dd6ce39a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:11.550430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b53d9b-7a14-4b28-8f7b-e4e92f927ef3 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:11.554539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9b38d-5207-416e-b106-d13d49929d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:11.555602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9b38d-5207-416e-b106-d13d49929d94 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:11.555734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9b38d-5207-416e-b106-d13d49929d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:11.556026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9b38d-5207-416e-b106-d13d49929d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:11.570494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96637ef6-6f62-4994-aeb5-3d8dd6ce39a4 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:11.575317 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d1028-4461-4a9b-b3d8-3ff677e4b6ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:11.576390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d1028-4461-4a9b-b3d8-3ff677e4b6ec service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:11.576390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d1028-4461-4a9b-b3d8-3ff677e4b6ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:11.576390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d1028-4461-4a9b-b3d8-3ff677e4b6ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:11.586787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d1028-4461-4a9b-b3d8-3ff677e4b6ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:11.588446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d1028-4461-4a9b-b3d8-3ff677e4b6ec service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:11.603875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1232/2467] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:11.633534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9b38d-5207-416e-b106-d13d49929d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:11.634979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9b38d-5207-416e-b106-d13d49929d94 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:11.644862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1236/2468] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:11.742312 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d1028-4461-4a9b-b3d8-3ff677e4b6ec service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:11.807996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d1028-4461-4a9b-b3d8-3ff677e4b6ec service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:11.819264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:11.823119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:11.823119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:11.823119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] PATH_INFO: `/v3/users/355fc0d588124aa5b775b05f8b34bfc4` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:11.823119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] RBAC: Authorizing `identity:delete_user(user_id=355fc0d588124aa5b775b05f8b34bfc4)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:11.827796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:11.901910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9b38d-5207-416e-b106-d13d49929d94 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:11.903435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e2f9b38d-5207-416e-b106-d13d49929d94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:11.969115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9b38d-5207-416e-b106-d13d49929d94 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:11.973777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68bfda-80cb-43bc-821a-8e66c5dd2079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:11.974435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68bfda-80cb-43bc-821a-8e66c5dd2079 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:11.974726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68bfda-80cb-43bc-821a-8e66c5dd2079 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:11.975000 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68bfda-80cb-43bc-821a-8e66c5dd2079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:11.976853 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ab68bfda-80cb-43bc-821a-8e66c5dd2079 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:11.989935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab68bfda-80cb-43bc-821a-8e66c5dd2079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:11.991378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab68bfda-80cb-43bc-821a-8e66c5dd2079 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:12.002019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1237/2469] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:12.037728 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '355fc0d588124aa5b775b05f8b34bfc4'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:12.038520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '355fc0d588124aa5b775b05f8b34bfc4'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:12.062177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '355fc0d588124aa5b775b05f8b34bfc4'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:12.072134 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '355fc0d588124aa5b775b05f8b34bfc4'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:12.196531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '355fc0d588124aa5b775b05f8b34bfc4'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:12.205910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1233/2470] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:11 2022] DELETE /identity/v3/users/355fc0d588124aa5b775b05f8b34bfc4 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:12.212936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081f3af-a0af-49b7-9281-342927e7dcb1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:12.213280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081f3af-a0af-49b7-9281-342927e7dcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:12.213588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081f3af-a0af-49b7-9281-342927e7dcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:12.279605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-1081f3af-a0af-49b7-9281-342927e7dcb1 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:12.281692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1081f3af-a0af-49b7-9281-342927e7dcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:12.397753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1238/2471] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:12.418863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:12.422429 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:12.451356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:12.498736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:12.512340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38f795-817c-4216-a294-9b75d3d369be admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:12.518940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358861f2-aa42-4425-afbb-fa084d2310f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:12.519745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358861f2-aa42-4425-afbb-fa084d2310f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:12.520048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358861f2-aa42-4425-afbb-fa084d2310f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:12.520522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358861f2-aa42-4425-afbb-fa084d2310f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:12.531177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358861f2-aa42-4425-afbb-fa084d2310f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:12.537880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358861f2-aa42-4425-afbb-fa084d2310f1 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:12.545462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1234/2472] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:12.941251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:13.015117 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:13.256425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081f3af-a0af-49b7-9281-342927e7dcb1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:13.258192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1081f3af-a0af-49b7-9281-342927e7dcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:13.281378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081f3af-a0af-49b7-9281-342927e7dcb1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:13.291506 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:13.292724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:13.293184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:13.293707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] PATH_INFO: `/v3/projects/1da1a758146842ba903abae94ec129c1` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:13.295071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da1a758146842ba903abae94ec129c1)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:13.297996 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:13.449678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da1a758146842ba903abae94ec129c1', 'request_id': 'req-45e7d340-e6bd-4a61-bb4c-147d1dd94051'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:13.567384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da1a758146842ba903abae94ec129c1', 'request_id': 'req-45e7d340-e6bd-4a61-bb4c-147d1dd94051'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:13.575324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da1a758146842ba903abae94ec129c1', 'request_id': 'req-45e7d340-e6bd-4a61-bb4c-147d1dd94051'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:13.588565 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] The token cache is being invalidate because project 1da1a758146842ba903abae94ec129c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:13.588851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:13.589593 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:13.591294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1239/2473] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:13 2022] DELETE /identity/v3/projects/1da1a758146842ba903abae94ec129c1 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:13.591748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:13.601120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358861f2-aa42-4425-afbb-fa084d2310f1 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:13.611085 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358861f2-aa42-4425-afbb-fa084d2310f1 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:13.641660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:13.683786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:13.684964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:13.685579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:13.686477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] PATH_INFO: `/v3/users/eb90ad7fea3b49c2b91639a7e8476b41` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:13.687946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb90ad7fea3b49c2b91639a7e8476b41)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:13.691691 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:13.922655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb90ad7fea3b49c2b91639a7e8476b41'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:13.923359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb90ad7fea3b49c2b91639a7e8476b41'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:13.945720 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb90ad7fea3b49c2b91639a7e8476b41'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:13.955329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb90ad7fea3b49c2b91639a7e8476b41'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:14.025696 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-014fd619-4082-45c7-9ee7-4c8a43ac7dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb90ad7fea3b49c2b91639a7e8476b41'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:14.041073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1235/2474] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:13 2022] DELETE /identity/v3/users/eb90ad7fea3b49c2b91639a7e8476b41 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:14.506986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:14.519248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:14.520817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:14.540189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7d340-e6bd-4a61-bb4c-147d1dd94051 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:14.620621 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:14.622933 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:14.623881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:14.624431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:14.625014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] PATH_INFO: `/v3/projects/326f2d3010294e2c9487409a298e464c` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:14.626537 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=326f2d3010294e2c9487409a298e464c)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:14.628821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:14.792621 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '326f2d3010294e2c9487409a298e464c', 'request_id': 'req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:14.952031 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '326f2d3010294e2c9487409a298e464c', 'request_id': 'req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:14.959864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '326f2d3010294e2c9487409a298e464c', 'request_id': 'req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:14.985224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] The token cache is being invalidate because project 326f2d3010294e2c9487409a298e464c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:14.985558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:14.986230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-eddf7f17-7e21-4442-9c7c-d3e6ff4cb38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:14.989272 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1240/2475] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:14 2022] DELETE /identity/v3/projects/326f2d3010294e2c9487409a298e464c => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:14.992779 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b571a-91e2-4892-95a0-7281fb25c73c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:14.993329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b571a-91e2-4892-95a0-7281fb25c73c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:14.993427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b571a-91e2-4892-95a0-7281fb25c73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:15.016162 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-0a0b571a-91e2-4892-95a0-7281fb25c73c None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:15.019521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0a0b571a-91e2-4892-95a0-7281fb25c73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:15.105445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1236/2476] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:51:15.116563 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25affe8-1863-4d04-994a-85bc38cc70b4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:15.120815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25affe8-1863-4d04-994a-85bc38cc70b4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:15.121093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25affe8-1863-4d04-994a-85bc38cc70b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:15.142131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e25affe8-1863-4d04-994a-85bc38cc70b4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:15.144302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e25affe8-1863-4d04-994a-85bc38cc70b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:15.153597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1241/2477] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:15.158552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b571a-91e2-4892-95a0-7281fb25c73c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:15.169846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b571a-91e2-4892-95a0-7281fb25c73c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:15.180177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007ae18-0198-4e49-bf06-c310f0600905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:15.181063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007ae18-0198-4e49-bf06-c310f0600905 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:15.181622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007ae18-0198-4e49-bf06-c310f0600905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:15.182238 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007ae18-0198-4e49-bf06-c310f0600905 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:15.183020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007ae18-0198-4e49-bf06-c310f0600905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:15.183327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007ae18-0198-4e49-bf06-c310f0600905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:15.185105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007ae18-0198-4e49-bf06-c310f0600905 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:15.199310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1237/2478] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:51:15.204252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25affe8-1863-4d04-994a-85bc38cc70b4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:15.217879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25affe8-1863-4d04-994a-85bc38cc70b4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:15.224382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5532b8-17ce-4241-b153-214499387b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:15.225797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5532b8-17ce-4241-b153-214499387b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:15.226132 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5532b8-17ce-4241-b153-214499387b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:15.227055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5532b8-17ce-4241-b153-214499387b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:15.227171 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5532b8-17ce-4241-b153-214499387b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:15.229220 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5532b8-17ce-4241-b153-214499387b38 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:15.360216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1242/2479] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:15 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:15.361952 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007ae18-0198-4e49-bf06-c310f0600905 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:15.373242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007ae18-0198-4e49-bf06-c310f0600905 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:15.447625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be82284-505c-4e5d-9f4f-0626bf92d0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:15.447625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be82284-505c-4e5d-9f4f-0626bf92d0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:15.447625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be82284-505c-4e5d-9f4f-0626bf92d0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:15.447625 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be82284-505c-4e5d-9f4f-0626bf92d0e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:15.448117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be82284-505c-4e5d-9f4f-0626bf92d0e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:15.450155 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be82284-505c-4e5d-9f4f-0626bf92d0e4 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:15.467125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:15.517396 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:15.573004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1238/2480] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:15 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:15.576974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5532b8-17ce-4241-b153-214499387b38 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:15.589786 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5532b8-17ce-4241-b153-214499387b38 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:15.592678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be82284-505c-4e5d-9f4f-0626bf92d0e4 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:15.595136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8be82284-505c-4e5d-9f4f-0626bf92d0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:15.599306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b3350-b371-4e8d-b873-0ea580157283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:15.600643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b3350-b371-4e8d-b873-0ea580157283 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:15.601078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b3350-b371-4e8d-b873-0ea580157283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:15.602645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b3350-b371-4e8d-b873-0ea580157283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:15.602645 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b3350-b371-4e8d-b873-0ea580157283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:15.604438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b3350-b371-4e8d-b873-0ea580157283 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:15.607467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be82284-505c-4e5d-9f4f-0626bf92d0e4 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:15.612942 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:15.613863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:15.614147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:15.614424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] PATH_INFO: `/v3/users/3fc4667a945946f1a0d9876843463cf7` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:15.615722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc4667a945946f1a0d9876843463cf7)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:15.619257 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1243/2481] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:51:15.619418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:15.620561 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b3350-b371-4e8d-b873-0ea580157283 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:15.631522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b3350-b371-4e8d-b873-0ea580157283 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:15.637683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585b3da-d6a7-4574-be6a-9f38b9bce267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:15.637974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585b3da-d6a7-4574-be6a-9f38b9bce267 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:15.638242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585b3da-d6a7-4574-be6a-9f38b9bce267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:15.638510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585b3da-d6a7-4574-be6a-9f38b9bce267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:15.639589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585b3da-d6a7-4574-be6a-9f38b9bce267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:15.640897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585b3da-d6a7-4574-be6a-9f38b9bce267 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:15.652861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1244/2482] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:15.658714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585b3da-d6a7-4574-be6a-9f38b9bce267 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:15.680868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585b3da-d6a7-4574-be6a-9f38b9bce267 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:15.687035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166cc72-4dce-4994-9e49-2fa65c504d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:15.687470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166cc72-4dce-4994-9e49-2fa65c504d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:15.687753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166cc72-4dce-4994-9e49-2fa65c504d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:15.688028 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166cc72-4dce-4994-9e49-2fa65c504d7e admin admin] PATH_INFO: `/v3/projects/9319e063b19341d5b5bdcc121363a476/users/d1d028c346ff4b67b4a3067a41cbd68f/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:15.721791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166cc72-4dce-4994-9e49-2fa65c504d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9319e063b19341d5b5bdcc121363a476, user_id=d1d028c346ff4b67b4a3067a41cbd68f, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:15.724729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166cc72-4dce-4994-9e49-2fa65c504d7e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:15.739118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:15.822748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1245/2483] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:15 2022] PUT /identity/v3/projects/9319e063b19341d5b5bdcc121363a476/users/d1d028c346ff4b67b4a3067a41cbd68f/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:15.828721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce853167-72b6-4291-9f70-88c898875b7b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:15.829680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce853167-72b6-4291-9f70-88c898875b7b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:15.830211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce853167-72b6-4291-9f70-88c898875b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:15.855863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc4667a945946f1a0d9876843463cf7'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:15.856551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc4667a945946f1a0d9876843463cf7'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:15.856960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-ce853167-72b6-4291-9f70-88c898875b7b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:15.863289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ce853167-72b6-4291-9f70-88c898875b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:15.895681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc4667a945946f1a0d9876843463cf7'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:15.913334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc4667a945946f1a0d9876843463cf7'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:15.964785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1246/2484] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:15.970490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce853167-72b6-4291-9f70-88c898875b7b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:15.971887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ce853167-72b6-4291-9f70-88c898875b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:15.989475 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc4667a945946f1a0d9876843463cf7'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:15.989913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce853167-72b6-4291-9f70-88c898875b7b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:15.996331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1239/2485] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:15 2022] DELETE /identity/v3/users/3fc4667a945946f1a0d9876843463cf7 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:15.999806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:15.999806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:15.999806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:15.999806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] PATH_INFO: `/v3/users/4e8b405029c34e92971172e2081d3915` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:16.000260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:16.000614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e8b405029c34e92971172e2081d3915)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:16.002139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:16.003244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:16.098912 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f1c17-947d-48f9-bba0-a4a008f5b384 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:16.102845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ab516-78a6-4b7e-8df6-5d7b7d2c2e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:16.103521 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ab516-78a6-4b7e-8df6-5d7b7d2c2e1a service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:16.103837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ab516-78a6-4b7e-8df6-5d7b7d2c2e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:16.104057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ab516-78a6-4b7e-8df6-5d7b7d2c2e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:16.173451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ab516-78a6-4b7e-8df6-5d7b7d2c2e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:16.175093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ab516-78a6-4b7e-8df6-5d7b7d2c2e1a service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:16.188441 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1240/2486] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:16.190402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d6259-56fe-414d-a73a-fe6be379c055 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:16.190855 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d6259-56fe-414d-a73a-fe6be379c055 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:16.191327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d6259-56fe-414d-a73a-fe6be379c055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:16.213643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-4b6d6259-56fe-414d-a73a-fe6be379c055 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:16.216286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4b6d6259-56fe-414d-a73a-fe6be379c055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:16.228545 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1241/2487] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:16.238614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e8b405029c34e92971172e2081d3915'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:16.242459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d6259-56fe-414d-a73a-fe6be379c055 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:16.246226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4b6d6259-56fe-414d-a73a-fe6be379c055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:16.259384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e8b405029c34e92971172e2081d3915'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:16.264119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d6259-56fe-414d-a73a-fe6be379c055 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:16.268945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fb8d2-3bda-4345-9a6f-83dbec8d1af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:16.270113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fb8d2-3bda-4345-9a6f-83dbec8d1af6 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:16.270568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fb8d2-3bda-4345-9a6f-83dbec8d1af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:16.271007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fb8d2-3bda-4345-9a6f-83dbec8d1af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:16.277293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e8b405029c34e92971172e2081d3915'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:16.284270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fb8d2-3bda-4345-9a6f-83dbec8d1af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:16.286208 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fb8d2-3bda-4345-9a6f-83dbec8d1af6 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:16.297009 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1242/2488] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:16.369407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e8b405029c34e92971172e2081d3915'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:16.370050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e8b405029c34e92971172e2081d3915'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:16.373192 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:16.379552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1247/2489] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:15 2022] DELETE /identity/v3/users/4e8b405029c34e92971172e2081d3915 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:16.383999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a87bb2-a879-4bb5-8cfc-74d896c08109 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:16.384294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a87bb2-a879-4bb5-8cfc-74d896c08109 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:16.384575 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a87bb2-a879-4bb5-8cfc-74d896c08109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:16.418114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-f8a87bb2-a879-4bb5-8cfc-74d896c08109 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:16.419607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f8a87bb2-a879-4bb5-8cfc-74d896c08109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:16.501050 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:16.542527 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1243/2490] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:16.558374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:16.560353 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:16.655951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d9f2-4e88-40a8-b507-d8102b949ab5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:16.661685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4deb8-0ccc-4c4d-b497-09d14413f27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:16.662596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4deb8-0ccc-4c4d-b497-09d14413f27a service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:16.662934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4deb8-0ccc-4c4d-b497-09d14413f27a service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:16.663327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4deb8-0ccc-4c4d-b497-09d14413f27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:16.678757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4deb8-0ccc-4c4d-b497-09d14413f27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:16.681329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4deb8-0ccc-4c4d-b497-09d14413f27a service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:16.696775 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1248/2491] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:17.123277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a87bb2-a879-4bb5-8cfc-74d896c08109 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:17.123277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f8a87bb2-a879-4bb5-8cfc-74d896c08109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:17.137774 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a87bb2-a879-4bb5-8cfc-74d896c08109 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:17.144552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:17.145814 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:17.150785 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:17.156761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] PATH_INFO: `/v3/projects/57f0359b89e040719cfcf0cba004adae` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:17.156761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f0359b89e040719cfcf0cba004adae)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:17.156761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:17.399844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f0359b89e040719cfcf0cba004adae', 'request_id': 'req-339e8f93-f409-467d-a6f5-cca4e5ac3781'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:17.422868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f0359b89e040719cfcf0cba004adae', 'request_id': 'req-339e8f93-f409-467d-a6f5-cca4e5ac3781'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:17.438267 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f0359b89e040719cfcf0cba004adae', 'request_id': 'req-339e8f93-f409-467d-a6f5-cca4e5ac3781'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:17.519415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] The token cache is being invalidate because project 57f0359b89e040719cfcf0cba004adae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:17.519951 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:17.521084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:17.524170 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1244/2492] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:17 2022] DELETE /identity/v3/projects/57f0359b89e040719cfcf0cba004adae => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:17.532531 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4deb8-0ccc-4c4d-b497-09d14413f27a service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:17.557076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4deb8-0ccc-4c4d-b497-09d14413f27a service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:17.664607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:17.665680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:17.667407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:17.667407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] PATH_INFO: `/v3/users/bef3933fda844a569102eae46a3038cc` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:17.667596 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef3933fda844a569102eae46a3038cc)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:17.669657 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:17.700757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:17.702588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:17.760641 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e8f93-f409-467d-a6f5-cca4e5ac3781 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:17.773015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:17.774488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:17.774786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:17.775071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] PATH_INFO: `/v3/projects/3c4213e0154f429fa1b268d98618ce1b` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:17.776846 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4213e0154f429fa1b268d98618ce1b)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:17.780327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:17.911394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef3933fda844a569102eae46a3038cc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:17.918283 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:17.919998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef3933fda844a569102eae46a3038cc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:17.936778 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef3933fda844a569102eae46a3038cc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.008999 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef3933fda844a569102eae46a3038cc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.009923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef3933fda844a569102eae46a3038cc'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.015892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4213e0154f429fa1b268d98618ce1b', 'request_id': 'req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.020728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1249/2493] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:17 2022] DELETE /identity/v3/users/bef3933fda844a569102eae46a3038cc => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:18.030868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4213e0154f429fa1b268d98618ce1b', 'request_id': 'req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.041123 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4213e0154f429fa1b268d98618ce1b', 'request_id': 'req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.104923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:18.140471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] The token cache is being invalidate because project 3c4213e0154f429fa1b268d98618ce1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:18.141057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.141514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.143293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1245/2494] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:17 2022] DELETE /identity/v3/projects/3c4213e0154f429fa1b268d98618ce1b => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:18.147568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:18.149011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:18.241988 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c85e4-0214-471e-87b0-4b7bc46a7576 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:18.244765 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:18.245716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:18.246242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:18.246527 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] PATH_INFO: `/v3/users/c93f93bd98a8411e8a94e69bef2d5ac1` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:18.247943 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93f93bd98a8411e8a94e69bef2d5ac1)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:18.250562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:18.485408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:18.489570 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:18.554424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93f93bd98a8411e8a94e69bef2d5ac1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.573884 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93f93bd98a8411e8a94e69bef2d5ac1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.599489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e0f75-79a7-4b9c-903a-d13e8a2b3014 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:18.611092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:18.611504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93f93bd98a8411e8a94e69bef2d5ac1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.612405 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:18.612841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:18.613327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] PATH_INFO: `/v3/users/899cbac79d874b308b500651bb40b2a7` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:18.616831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=899cbac79d874b308b500651bb40b2a7)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:18.620060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:18.903573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93f93bd98a8411e8a94e69bef2d5ac1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.904150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93f93bd98a8411e8a94e69bef2d5ac1'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:18.910539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1250/2495] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:18 2022] DELETE /identity/v3/users/c93f93bd98a8411e8a94e69bef2d5ac1 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:18.966917 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:18.969651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:19.063699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904a516-edcd-40ef-8abe-7d4250df3951 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:19.080954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:19.082765 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:19.083725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:19.084327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '899cbac79d874b308b500651bb40b2a7'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.084973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] PATH_INFO: `/v3/users/d352cf5c00ae450c9a7ca4dcc68a87db` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:19.085244 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '899cbac79d874b308b500651bb40b2a7'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.086311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d352cf5c00ae450c9a7ca4dcc68a87db)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:19.089746 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:19.101357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '899cbac79d874b308b500651bb40b2a7'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.109506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '899cbac79d874b308b500651bb40b2a7'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.225583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '899cbac79d874b308b500651bb40b2a7'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.234632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1246/2496] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:18 2022] DELETE /identity/v3/users/899cbac79d874b308b500651bb40b2a7 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:19.236894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:19.243824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:19.321906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223739b-5270-47fe-8ee0-5848450a6c66 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:19.328409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:19.329071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:19.329592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:19.329969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] PATH_INFO: `/v3/projects/81f509c9dc3e45aca270e298fdfbcf3f` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:19.332384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81f509c9dc3e45aca270e298fdfbcf3f)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:19.334718 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:19.384244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd352cf5c00ae450c9a7ca4dcc68a87db'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.391197 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd352cf5c00ae450c9a7ca4dcc68a87db'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.404551 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd352cf5c00ae450c9a7ca4dcc68a87db'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.575467 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd352cf5c00ae450c9a7ca4dcc68a87db'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.577852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-907b5a92-008f-4e1f-aef8-b6a5b99874ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd352cf5c00ae450c9a7ca4dcc68a87db'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.592562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1251/2497] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:18 2022] DELETE /identity/v3/users/d352cf5c00ae450c9a7ca4dcc68a87db => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:19.594466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d437235-8fdf-4ff8-8696-9a78464e2c5f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:19.594925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d437235-8fdf-4ff8-8696-9a78464e2c5f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:19.595476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d437235-8fdf-4ff8-8696-9a78464e2c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:19.602973 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81f509c9dc3e45aca270e298fdfbcf3f', 'request_id': 'req-65558c7e-976d-4988-8409-c4bb57330db5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.610426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81f509c9dc3e45aca270e298fdfbcf3f', 'request_id': 'req-65558c7e-976d-4988-8409-c4bb57330db5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.617811 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81f509c9dc3e45aca270e298fdfbcf3f', 'request_id': 'req-65558c7e-976d-4988-8409-c4bb57330db5'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.641262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8d437235-8fdf-4ff8-8696-9a78464e2c5f None None] MFA Rules not processed for user `d1d028c346ff4b67b4a3067a41cbd68f`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:19.643172 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8d437235-8fdf-4ff8-8696-9a78464e2c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:19.712498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1252/2498] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:19.722070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d437235-8fdf-4ff8-8696-9a78464e2c5f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:19.723727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8d437235-8fdf-4ff8-8696-9a78464e2c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:19.767955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] The token cache is being invalidate because project 81f509c9dc3e45aca270e298fdfbcf3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:19.768543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.769266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:19.773202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1247/2499] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:19 2022] DELETE /identity/v3/projects/81f509c9dc3e45aca270e298fdfbcf3f => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:19.773202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:19.774085 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:19.799219 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d437235-8fdf-4ff8-8696-9a78464e2c5f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:19.804731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:19.805265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:19.805547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:19.805822 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] PATH_INFO: `/v3/projects/d5691d4e7b454065be1b217b5ffb001d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:19.807081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5691d4e7b454065be1b217b5ffb001d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:19.808980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:19.844694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65558c7e-976d-4988-8409-c4bb57330db5 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:19.849409 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:19.850368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:19.850661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:19.850934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] PATH_INFO: `/v3/projects/81777430895b432999f69033667e59f1` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:19.852139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81777430895b432999f69033667e59f1)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:19.854101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:19.966823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5691d4e7b454065be1b217b5ffb001d', 'request_id': 'req-a0e164e5-9185-4721-a72c-c95f4cc0fb56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.001402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81777430895b432999f69033667e59f1', 'request_id': 'req-97c1d298-9171-453c-bc9d-39f53b71f867'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.008741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81777430895b432999f69033667e59f1', 'request_id': 'req-97c1d298-9171-453c-bc9d-39f53b71f867'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.014225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81777430895b432999f69033667e59f1', 'request_id': 'req-97c1d298-9171-453c-bc9d-39f53b71f867'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.070258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5691d4e7b454065be1b217b5ffb001d', 'request_id': 'req-a0e164e5-9185-4721-a72c-c95f4cc0fb56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.076655 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5691d4e7b454065be1b217b5ffb001d', 'request_id': 'req-a0e164e5-9185-4721-a72c-c95f4cc0fb56'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.087832 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] The token cache is being invalidate because project d5691d4e7b454065be1b217b5ffb001d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:20.088119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.088780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.090355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1253/2500] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:19 2022] DELETE /identity/v3/projects/d5691d4e7b454065be1b217b5ffb001d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:20.091201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.092476 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.156828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e164e5-9185-4721-a72c-c95f4cc0fb56 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.160778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] The token cache is being invalidate because project 81777430895b432999f69033667e59f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:20.161093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.161770 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-97c1d298-9171-453c-bc9d-39f53b71f867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.163337 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1248/2501] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:19 2022] DELETE /identity/v3/projects/81777430895b432999f69033667e59f1 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:20.165539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bf7a0-d874-4596-861c-950ef7672977 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.165830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bf7a0-d874-4596-861c-950ef7672977 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.166111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bf7a0-d874-4596-861c-950ef7672977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.185675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-481bf7a0-d874-4596-861c-950ef7672977 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:20.186875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-481bf7a0-d874-4596-861c-950ef7672977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.215964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c13c2-032d-4427-9c98-7cd1a3daa425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:20.216631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c13c2-032d-4427-9c98-7cd1a3daa425 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.216917 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c13c2-032d-4427-9c98-7cd1a3daa425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.217225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c13c2-032d-4427-9c98-7cd1a3daa425 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.217925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c13c2-032d-4427-9c98-7cd1a3daa425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:20.219831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c13c2-032d-4427-9c98-7cd1a3daa425 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:20.245076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1249/2502] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:20.246061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bf7a0-d874-4596-861c-950ef7672977 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.247393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-481bf7a0-d874-4596-861c-950ef7672977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.259168 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bf7a0-d874-4596-861c-950ef7672977 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.263698 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:20.264544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.264823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.265093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] PATH_INFO: `/v3/projects/ca23768669a74586a221c14ebaa8ee26` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.266299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca23768669a74586a221c14ebaa8ee26)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:20.268182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:20.302588 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1254/2503] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:20 2022] POST /identity/v3/projects => generated 371 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:20.303924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b12d72-3b7c-4309-8497-01c707aeb981 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.304223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b12d72-3b7c-4309-8497-01c707aeb981 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.304498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b12d72-3b7c-4309-8497-01c707aeb981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.321983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-57b12d72-3b7c-4309-8497-01c707aeb981 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:20.323182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-57b12d72-3b7c-4309-8497-01c707aeb981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.368626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca23768669a74586a221c14ebaa8ee26', 'request_id': 'req-2022a7fd-46e0-450a-8625-b222a6232a62'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.375333 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca23768669a74586a221c14ebaa8ee26', 'request_id': 'req-2022a7fd-46e0-450a-8625-b222a6232a62'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.382401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca23768669a74586a221c14ebaa8ee26', 'request_id': 'req-2022a7fd-46e0-450a-8625-b222a6232a62'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.384370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1255/2504] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:20.385684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093415cc-d582-48e0-bb9c-e33d94105411 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.385982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093415cc-d582-48e0-bb9c-e33d94105411 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.386265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093415cc-d582-48e0-bb9c-e33d94105411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.406736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-093415cc-d582-48e0-bb9c-e33d94105411 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:20.410571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-093415cc-d582-48e0-bb9c-e33d94105411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.429948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:20.469595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] The token cache is being invalidate because project ca23768669a74586a221c14ebaa8ee26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:20.469878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.470555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-2022a7fd-46e0-450a-8625-b222a6232a62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.472679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1250/2505] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:20 2022] DELETE /identity/v3/projects/ca23768669a74586a221c14ebaa8ee26 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:20.474757 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1256/2506] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:20.475825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093415cc-d582-48e0-bb9c-e33d94105411 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.476877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa07d0-0c80-4f68-9c0a-2e6bd460b0b1 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.477188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa07d0-0c80-4f68-9c0a-2e6bd460b0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.477373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-093415cc-d582-48e0-bb9c-e33d94105411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.477531 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa07d0-0c80-4f68-9c0a-2e6bd460b0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.487521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:20.491516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093415cc-d582-48e0-bb9c-e33d94105411 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.496400 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84392691-dac9-476a-9b49-2602aa0da494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:20.497073 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84392691-dac9-476a-9b49-2602aa0da494 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.497389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84392691-dac9-476a-9b49-2602aa0da494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.497668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84392691-dac9-476a-9b49-2602aa0da494 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.498389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84392691-dac9-476a-9b49-2602aa0da494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:20.500310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84392691-dac9-476a-9b49-2602aa0da494 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:20.500437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-51aa07d0-0c80-4f68-9c0a-2e6bd460b0b1 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:20.501613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-51aa07d0-0c80-4f68-9c0a-2e6bd460b0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.510732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1251/2507] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:20.511994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32326d9f-8266-4f58-bc45-3e60f6b92817 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.512275 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32326d9f-8266-4f58-bc45-3e60f6b92817 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.512547 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32326d9f-8266-4f58-bc45-3e60f6b92817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.534735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-32326d9f-8266-4f58-bc45-3e60f6b92817 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:20.535932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-32326d9f-8266-4f58-bc45-3e60f6b92817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.543910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1252/2508] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:20.544694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32326d9f-8266-4f58-bc45-3e60f6b92817 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.546000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-32326d9f-8266-4f58-bc45-3e60f6b92817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.557647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32326d9f-8266-4f58-bc45-3e60f6b92817 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.562130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:20.562972 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.563246 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.563517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] PATH_INFO: `/v3/users/ad07006492bb407da09a6b5387237f55` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.564706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad07006492bb407da09a6b5387237f55)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:20.566701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:20.582304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1257/2509] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:20 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:20.582304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2b0df-38d3-4df1-802a-412c8c0867ad None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.582304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2b0df-38d3-4df1-802a-412c8c0867ad None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.582304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2b0df-38d3-4df1-802a-412c8c0867ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.600688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-75d2b0df-38d3-4df1-802a-412c8c0867ad None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:20.601930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-75d2b0df-38d3-4df1-802a-412c8c0867ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.610081 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1258/2510] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:20.613249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2b0df-38d3-4df1-802a-412c8c0867ad None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.620144 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2b0df-38d3-4df1-802a-412c8c0867ad None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.624743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d20ca8-d403-4b39-93cd-e8322d2758d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:20.625407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d20ca8-d403-4b39-93cd-e8322d2758d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.625690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d20ca8-d403-4b39-93cd-e8322d2758d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.625964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d20ca8-d403-4b39-93cd-e8322d2758d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.626712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d20ca8-d403-4b39-93cd-e8322d2758d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:20.627015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d20ca8-d403-4b39-93cd-e8322d2758d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:20.628575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d20ca8-d403-4b39-93cd-e8322d2758d4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:20.640478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1259/2511] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:51:20.641389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d20ca8-d403-4b39-93cd-e8322d2758d4 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.651449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d20ca8-d403-4b39-93cd-e8322d2758d4 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.656118 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb996c7-a600-4187-8ad1-6a65f96a926b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:20.656889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb996c7-a600-4187-8ad1-6a65f96a926b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.657250 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb996c7-a600-4187-8ad1-6a65f96a926b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.660771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb996c7-a600-4187-8ad1-6a65f96a926b admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.660771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb996c7-a600-4187-8ad1-6a65f96a926b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:20.660771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb996c7-a600-4187-8ad1-6a65f96a926b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:20.661056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb996c7-a600-4187-8ad1-6a65f96a926b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:20.673056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1260/2512] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:51:20.674526 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb996c7-a600-4187-8ad1-6a65f96a926b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.676518 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9bb996c7-a600-4187-8ad1-6a65f96a926b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.688995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb996c7-a600-4187-8ad1-6a65f96a926b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.693631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22232-ed23-4888-b74c-f411cb654a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:20.694306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22232-ed23-4888-b74c-f411cb654a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.694589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22232-ed23-4888-b74c-f411cb654a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.694861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22232-ed23-4888-b74c-f411cb654a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.695581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e22232-ed23-4888-b74c-f411cb654a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:20.697143 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e22232-ed23-4888-b74c-f411cb654a91 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:20.707732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1261/2513] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:20.708562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22232-ed23-4888-b74c-f411cb654a91 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.718340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22232-ed23-4888-b74c-f411cb654a91 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.723803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bdb32-3adf-4c07-812a-67d3fe8aeb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:20.724484 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bdb32-3adf-4c07-812a-67d3fe8aeb81 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.724773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bdb32-3adf-4c07-812a-67d3fe8aeb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.725045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bdb32-3adf-4c07-812a-67d3fe8aeb81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.725824 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bdb32-3adf-4c07-812a-67d3fe8aeb81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:20.726135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bdb32-3adf-4c07-812a-67d3fe8aeb81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:20.727737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bdb32-3adf-4c07-812a-67d3fe8aeb81 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:20.743063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad07006492bb407da09a6b5387237f55'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.746882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1262/2514] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:51:20.746882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bdb32-3adf-4c07-812a-67d3fe8aeb81 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.747229 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad07006492bb407da09a6b5387237f55'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.755917 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bdb32-3adf-4c07-812a-67d3fe8aeb81 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.759352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad07006492bb407da09a6b5387237f55'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.761705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739b87e-c22c-4302-b19d-6a0dc3e8f508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:20.762368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739b87e-c22c-4302-b19d-6a0dc3e8f508 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:20.762659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739b87e-c22c-4302-b19d-6a0dc3e8f508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:20.762936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739b87e-c22c-4302-b19d-6a0dc3e8f508 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:20.763627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739b87e-c22c-4302-b19d-6a0dc3e8f508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:20.765752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739b87e-c22c-4302-b19d-6a0dc3e8f508 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:20.766569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad07006492bb407da09a6b5387237f55'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.854158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad07006492bb407da09a6b5387237f55'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:20.860631 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1253/2515] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:20 2022] DELETE /identity/v3/users/ad07006492bb407da09a6b5387237f55 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:20.862112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.863464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.915194 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1263/2516] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:20 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:20.919472 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739b87e-c22c-4302-b19d-6a0dc3e8f508 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:20.921195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5739b87e-c22c-4302-b19d-6a0dc3e8f508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:20.929480 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f00a33-fc79-405e-9d07-0f5f67cd8b55 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.993664 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739b87e-c22c-4302-b19d-6a0dc3e8f508 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:20.999630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf571b-307f-4b72-9490-15b63f5c2151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.000428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf571b-307f-4b72-9490-15b63f5c2151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.000734 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf571b-307f-4b72-9490-15b63f5c2151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.000992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf571b-307f-4b72-9490-15b63f5c2151 admin admin] PATH_INFO: `/v3/projects/ba91797bcd2a4c62855828e00c7053ed/users/4a87961bce7e4333bbe1dc29dfce9617/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.002569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a288c5-74a2-4855-956d-558d8940292f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.003508 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a288c5-74a2-4855-956d-558d8940292f admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.003960 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a288c5-74a2-4855-956d-558d8940292f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.004395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a288c5-74a2-4855-956d-558d8940292f admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.005281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a288c5-74a2-4855-956d-558d8940292f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.007377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a288c5-74a2-4855-956d-558d8940292f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.017939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf571b-307f-4b72-9490-15b63f5c2151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba91797bcd2a4c62855828e00c7053ed, user_id=4a87961bce7e4333bbe1dc29dfce9617, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.020592 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf571b-307f-4b72-9490-15b63f5c2151 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.119468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1254/2517] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:20 2022] POST /identity/v3/projects => generated 385 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:21.120207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a288c5-74a2-4855-956d-558d8940292f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.121839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-67a288c5-74a2-4855-956d-558d8940292f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:21.152467 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1264/2518] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:20 2022] PUT /identity/v3/projects/ba91797bcd2a4c62855828e00c7053ed/users/4a87961bce7e4333bbe1dc29dfce9617/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:21.153456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf571b-307f-4b72-9490-15b63f5c2151 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.155394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-35bf571b-307f-4b72-9490-15b63f5c2151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:21.194294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a288c5-74a2-4855-956d-558d8940292f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.231911 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf571b-307f-4b72-9490-15b63f5c2151 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.238972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a94032-142c-4248-87ef-b66074553939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.239685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a94032-142c-4248-87ef-b66074553939 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.242079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a94032-142c-4248-87ef-b66074553939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.242079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a94032-142c-4248-87ef-b66074553939 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.242079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a94032-142c-4248-87ef-b66074553939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.242904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a94032-142c-4248-87ef-b66074553939 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.251205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f662df-3564-4e3d-8511-ee1df973e44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.252430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f662df-3564-4e3d-8511-ee1df973e44e admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.252716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f662df-3564-4e3d-8511-ee1df973e44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.252987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f662df-3564-4e3d-8511-ee1df973e44e admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.253775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f662df-3564-4e3d-8511-ee1df973e44e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.255686 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f662df-3564-4e3d-8511-ee1df973e44e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.347009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1265/2519] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:21 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:21.347009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a94032-142c-4248-87ef-b66074553939 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.354780 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a94032-142c-4248-87ef-b66074553939 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.362131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1255/2520] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:21 2022] POST /identity/v3/projects => generated 373 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:21.362131 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f662df-3564-4e3d-8511-ee1df973e44e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.362380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33063bbf-3441-4509-9e4a-65b5476c0922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.362591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-41f662df-3564-4e3d-8511-ee1df973e44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:21.362688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33063bbf-3441-4509-9e4a-65b5476c0922 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.362688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33063bbf-3441-4509-9e4a-65b5476c0922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.362688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33063bbf-3441-4509-9e4a-65b5476c0922 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.362688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33063bbf-3441-4509-9e4a-65b5476c0922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.363834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33063bbf-3441-4509-9e4a-65b5476c0922 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.407056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:21.427692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f662df-3564-4e3d-8511-ee1df973e44e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.434018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.434018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.434018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.434395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] PATH_INFO: `/v3/projects/a0070026dba04010b93def3ca14598ea` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.435443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0070026dba04010b93def3ca14598ea)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.437580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.444182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1266/2521] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:21 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:21.448262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33063bbf-3441-4509-9e4a-65b5476c0922 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.458516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33063bbf-3441-4509-9e4a-65b5476c0922 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.463317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cfe29-41d1-4c06-a133-9a860ac0d5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.464040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cfe29-41d1-4c06-a133-9a860ac0d5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.465738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cfe29-41d1-4c06-a133-9a860ac0d5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.465738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cfe29-41d1-4c06-a133-9a860ac0d5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.465738 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020cfe29-41d1-4c06-a133-9a860ac0d5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.466941 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020cfe29-41d1-4c06-a133-9a860ac0d5b8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.477989 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1267/2522] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:21.478848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cfe29-41d1-4c06-a133-9a860ac0d5b8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.488840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cfe29-41d1-4c06-a133-9a860ac0d5b8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.493710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd921a-b625-4c2f-b61a-fb9be6d19c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.494366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd921a-b625-4c2f-b61a-fb9be6d19c48 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.494651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd921a-b625-4c2f-b61a-fb9be6d19c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.494950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd921a-b625-4c2f-b61a-fb9be6d19c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.495671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd921a-b625-4c2f-b61a-fb9be6d19c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.497340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd921a-b625-4c2f-b61a-fb9be6d19c48 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.508703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1268/2523] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:21.509514 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd921a-b625-4c2f-b61a-fb9be6d19c48 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.518874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd921a-b625-4c2f-b61a-fb9be6d19c48 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.523897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c5593-1727-48a3-9a88-f702d2d2ad1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.524539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c5593-1727-48a3-9a88-f702d2d2ad1f admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.524825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c5593-1727-48a3-9a88-f702d2d2ad1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.525103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c5593-1727-48a3-9a88-f702d2d2ad1f admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.525818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c5593-1727-48a3-9a88-f702d2d2ad1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.527848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c5593-1727-48a3-9a88-f702d2d2ad1f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.591564 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0070026dba04010b93def3ca14598ea', 'request_id': 'req-efb2d902-9535-4dda-994c-9dff41dff175'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:21.598347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0070026dba04010b93def3ca14598ea', 'request_id': 'req-efb2d902-9535-4dda-994c-9dff41dff175'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:21.604215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0070026dba04010b93def3ca14598ea', 'request_id': 'req-efb2d902-9535-4dda-994c-9dff41dff175'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:21.610971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1269/2524] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:21 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:21.610971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c5593-1727-48a3-9a88-f702d2d2ad1f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.620120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c5593-1727-48a3-9a88-f702d2d2ad1f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.675143 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e210910-18b9-4d1b-a297-429a295e0865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.675739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e210910-18b9-4d1b-a297-429a295e0865 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.676040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e210910-18b9-4d1b-a297-429a295e0865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.676284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e210910-18b9-4d1b-a297-429a295e0865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.677001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e210910-18b9-4d1b-a297-429a295e0865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.678605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e210910-18b9-4d1b-a297-429a295e0865 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.688262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1270/2525] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:21.689033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e210910-18b9-4d1b-a297-429a295e0865 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.698673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e210910-18b9-4d1b-a297-429a295e0865 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.703427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba6e39-042f-48dc-969b-9fa64f6fd2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.704303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba6e39-042f-48dc-969b-9fa64f6fd2cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.704702 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba6e39-042f-48dc-969b-9fa64f6fd2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.704985 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba6e39-042f-48dc-969b-9fa64f6fd2cf admin admin] PATH_INFO: `/v3/projects/ba91797bcd2a4c62855828e00c7053ed/users/4a87961bce7e4333bbe1dc29dfce9617/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.706998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba6e39-042f-48dc-969b-9fa64f6fd2cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba91797bcd2a4c62855828e00c7053ed, user_id=4a87961bce7e4333bbe1dc29dfce9617, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.709642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba6e39-042f-48dc-969b-9fa64f6fd2cf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.721234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1271/2526] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:21 2022] PUT /identity/v3/projects/ba91797bcd2a4c62855828e00c7053ed/users/4a87961bce7e4333bbe1dc29dfce9617/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:21.722027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba6e39-042f-48dc-969b-9fa64f6fd2cf admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.725943 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] The token cache is being invalidate because project a0070026dba04010b93def3ca14598ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:21.728172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:21.728172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:21.730165 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1256/2527] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:21 2022] DELETE /identity/v3/projects/a0070026dba04010b93def3ca14598ea => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:21.731262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.733532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:21.750477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2d902-9535-4dda-994c-9dff41dff175 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.786807 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba6e39-042f-48dc-969b-9fa64f6fd2cf admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.791688 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02d27f-bbe1-4d09-ad76-71d2aa08099a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.792449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02d27f-bbe1-4d09-ad76-71d2aa08099a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.792803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02d27f-bbe1-4d09-ad76-71d2aa08099a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.793076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02d27f-bbe1-4d09-ad76-71d2aa08099a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.793849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02d27f-bbe1-4d09-ad76-71d2aa08099a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.795564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02d27f-bbe1-4d09-ad76-71d2aa08099a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.806480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1272/2528] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:21.807326 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02d27f-bbe1-4d09-ad76-71d2aa08099a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.810865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9c02d27f-bbe1-4d09-ad76-71d2aa08099a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:21.813848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1ac1f-e772-4b09-a1ff-a7d7aca18fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.814901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1ac1f-e772-4b09-a1ff-a7d7aca18fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.815176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1ac1f-e772-4b09-a1ff-a7d7aca18fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.815452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1ac1f-e772-4b09-a1ff-a7d7aca18fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.816186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1ac1f-e772-4b09-a1ff-a7d7aca18fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.817829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1ac1f-e772-4b09-a1ff-a7d7aca18fb0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.823358 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02d27f-bbe1-4d09-ad76-71d2aa08099a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.829136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1257/2529] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:21.829283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92ea00-3349-4d05-9940-4a065287f835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.829283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92ea00-3349-4d05-9940-4a065287f835 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.829283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92ea00-3349-4d05-9940-4a065287f835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.829283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92ea00-3349-4d05-9940-4a065287f835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.829671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082ca1f-5da9-4997-81b5-5881adedb947 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.829671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082ca1f-5da9-4997-81b5-5881adedb947 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.829832 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082ca1f-5da9-4997-81b5-5881adedb947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.829913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92ea00-3349-4d05-9940-4a065287f835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.831399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92ea00-3349-4d05-9940-4a065287f835 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.845034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1273/2530] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:21.845034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92ea00-3349-4d05-9940-4a065287f835 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.845034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eb92ea00-3349-4d05-9940-4a065287f835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:21.848442 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-4082ca1f-5da9-4997-81b5-5881adedb947 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:21.849670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4082ca1f-5da9-4997-81b5-5881adedb947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:21.858574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92ea00-3349-4d05-9940-4a065287f835 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.862014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1258/2531] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:21.863057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082ca1f-5da9-4997-81b5-5881adedb947 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.867670 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd100a7-df57-47ef-971e-32e67ac4b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.868840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd100a7-df57-47ef-971e-32e67ac4b39d admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.869633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd100a7-df57-47ef-971e-32e67ac4b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.870147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd100a7-df57-47ef-971e-32e67ac4b39d admin admin] PATH_INFO: `/v3/projects/1b1b092e5cbb435ba9b9e6c0b2824cdf/users/057fd9ecc91d40edbd05da84e22fafa8/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.873734 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082ca1f-5da9-4997-81b5-5881adedb947 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.879909 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b70ca-c5ac-4c7e-8ef0-041d6cdf8b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.880742 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b70ca-c5ac-4c7e-8ef0-041d6cdf8b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.881122 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b70ca-c5ac-4c7e-8ef0-041d6cdf8b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.881533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b70ca-c5ac-4c7e-8ef0-041d6cdf8b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.882383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b70ca-c5ac-4c7e-8ef0-041d6cdf8b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.884452 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b70ca-c5ac-4c7e-8ef0-041d6cdf8b4e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.890924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd100a7-df57-47ef-971e-32e67ac4b39d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b1b092e5cbb435ba9b9e6c0b2824cdf, user_id=057fd9ecc91d40edbd05da84e22fafa8, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.894568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd100a7-df57-47ef-971e-32e67ac4b39d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.898513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1259/2532] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:21.898513 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b70ca-c5ac-4c7e-8ef0-041d6cdf8b4e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:21.910745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b70ca-c5ac-4c7e-8ef0-041d6cdf8b4e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:21.917701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6291f8-331a-40f9-9aa2-549305d4d961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:21.918340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6291f8-331a-40f9-9aa2-549305d4d961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:21.918640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6291f8-331a-40f9-9aa2-549305d4d961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:21.918902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6291f8-331a-40f9-9aa2-549305d4d961 admin admin] PATH_INFO: `/v3/projects/47770a29807b44ba8c74c2b7d3b47a83/users/72f9a39f2899442cb20514bff3c46dda/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:21.932782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:21.935697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6291f8-331a-40f9-9aa2-549305d4d961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47770a29807b44ba8c74c2b7d3b47a83, user_id=72f9a39f2899442cb20514bff3c46dda, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:21.938425 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6291f8-331a-40f9-9aa2-549305d4d961 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:21.980831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1274/2533] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:21 2022] PUT /identity/v3/projects/1b1b092e5cbb435ba9b9e6c0b2824cdf/users/057fd9ecc91d40edbd05da84e22fafa8/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:21.981817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd100a7-df57-47ef-971e-32e67ac4b39d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:22.031998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1260/2534] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:21 2022] PUT /identity/v3/projects/47770a29807b44ba8c74c2b7d3b47a83/users/72f9a39f2899442cb20514bff3c46dda/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:22.033903 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955c6a8-5e6c-4d23-8abb-f7715a9c9be6 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.034243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955c6a8-5e6c-4d23-8abb-f7715a9c9be6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.034585 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955c6a8-5e6c-4d23-8abb-f7715a9c9be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.055401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd100a7-df57-47ef-971e-32e67ac4b39d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:22.057027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-8955c6a8-5e6c-4d23-8abb-f7715a9c9be6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:22.058293 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8955c6a8-5e6c-4d23-8abb-f7715a9c9be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:22.116054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34145597-c23a-43f3-983a-5c133268b515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:22.117598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34145597-c23a-43f3-983a-5c133268b515 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.117598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34145597-c23a-43f3-983a-5c133268b515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.117598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34145597-c23a-43f3-983a-5c133268b515 admin admin] PATH_INFO: `/v3/projects/0f68263b01c64cbd8c759ba840c949f7/users/0db86d876f484602abe6964ea79c8da8/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.122424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1261/2535] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:22.124004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805c273-7d66-4e5b-9d14-aecdecaf4026 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.124292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805c273-7d66-4e5b-9d14-aecdecaf4026 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.124717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805c273-7d66-4e5b-9d14-aecdecaf4026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.137008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34145597-c23a-43f3-983a-5c133268b515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f68263b01c64cbd8c759ba840c949f7, user_id=0db86d876f484602abe6964ea79c8da8, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:22.139922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34145597-c23a-43f3-983a-5c133268b515 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:22.153099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-b805c273-7d66-4e5b-9d14-aecdecaf4026 None None] MFA Rules not processed for user `057fd9ecc91d40edbd05da84e22fafa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:22.154573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b805c273-7d66-4e5b-9d14-aecdecaf4026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:22.218831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1262/2536] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:22.219875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72705b2e-be36-4c8a-a38c-e52bbf2507f6 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.220160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72705b2e-be36-4c8a-a38c-e52bbf2507f6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.221101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72705b2e-be36-4c8a-a38c-e52bbf2507f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.221384 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1275/2537] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:21 2022] PUT /identity/v3/projects/0f68263b01c64cbd8c759ba840c949f7/users/0db86d876f484602abe6964ea79c8da8/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:22.222160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34145597-c23a-43f3-983a-5c133268b515 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:22.247537 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-72705b2e-be36-4c8a-a38c-e52bbf2507f6 None None] MFA Rules not processed for user `72f9a39f2899442cb20514bff3c46dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:22.248745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-72705b2e-be36-4c8a-a38c-e52bbf2507f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:22.287895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34145597-c23a-43f3-983a-5c133268b515 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:22.294099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abbe94-85c0-400a-84e1-4bfe0e57d20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:22.303047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abbe94-85c0-400a-84e1-4bfe0e57d20b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.303047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abbe94-85c0-400a-84e1-4bfe0e57d20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.303047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abbe94-85c0-400a-84e1-4bfe0e57d20b admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.303047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abbe94-85c0-400a-84e1-4bfe0e57d20b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:22.303047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abbe94-85c0-400a-84e1-4bfe0e57d20b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:22.303047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abbe94-85c0-400a-84e1-4bfe0e57d20b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:22.310164 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1276/2538] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:51:22.311448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e50754-b305-4d9d-a202-10478bc7c525 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.311733 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e50754-b305-4d9d-a202-10478bc7c525 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.312211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e50754-b305-4d9d-a202-10478bc7c525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.318210 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1263/2539] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:22.319789 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72705b2e-be36-4c8a-a38c-e52bbf2507f6 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:22.333087 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72705b2e-be36-4c8a-a38c-e52bbf2507f6 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:22.339025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c219f12-8e22-4fa9-b1aa-74a85cb967fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:22.339758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c219f12-8e22-4fa9-b1aa-74a85cb967fb admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.340050 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c219f12-8e22-4fa9-b1aa-74a85cb967fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.340328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c219f12-8e22-4fa9-b1aa-74a85cb967fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.341034 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c219f12-8e22-4fa9-b1aa-74a85cb967fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:22.342598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f6e50754-b305-4d9d-a202-10478bc7c525 None None] MFA Rules not processed for user `0db86d876f484602abe6964ea79c8da8`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:22.343036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c219f12-8e22-4fa9-b1aa-74a85cb967fb admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:22.343801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f6e50754-b305-4d9d-a202-10478bc7c525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:22.411404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1277/2540] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:22.412251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e50754-b305-4d9d-a202-10478bc7c525 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:22.423148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e50754-b305-4d9d-a202-10478bc7c525 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:22.427827 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45e521-f289-4273-971f-64fd1c5c3826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:22.428492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45e521-f289-4273-971f-64fd1c5c3826 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.429256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45e521-f289-4273-971f-64fd1c5c3826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.429256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45e521-f289-4273-971f-64fd1c5c3826 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.429743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45e521-f289-4273-971f-64fd1c5c3826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:22.431692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45e521-f289-4273-971f-64fd1c5c3826 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:22.495825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1264/2541] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:22 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:22.496550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c219f12-8e22-4fa9-b1aa-74a85cb967fb admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:22.561292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c219f12-8e22-4fa9-b1aa-74a85cb967fb admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:22.567444 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7d989-5e6e-4979-b5ac-61afb1135bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:22.567872 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7d989-5e6e-4979-b5ac-61afb1135bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.568161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7d989-5e6e-4979-b5ac-61afb1135bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.568441 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7d989-5e6e-4979-b5ac-61afb1135bf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.569140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7d989-5e6e-4979-b5ac-61afb1135bf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:22.574583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1278/2542] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:22 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:22.574753 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7d989-5e6e-4979-b5ac-61afb1135bf1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:22.574880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45e521-f289-4273-971f-64fd1c5c3826 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:22.575987 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ac45e521-f289-4273-971f-64fd1c5c3826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:22.650790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45e521-f289-4273-971f-64fd1c5c3826 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:22.656471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6654b3-6537-49a2-b24b-915ea70563c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:22.657128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6654b3-6537-49a2-b24b-915ea70563c8 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.657446 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6654b3-6537-49a2-b24b-915ea70563c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.657725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6654b3-6537-49a2-b24b-915ea70563c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.696424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1265/2543] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:22 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:22.696910 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7d989-5e6e-4979-b5ac-61afb1135bf1 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:22.717661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6654b3-6537-49a2-b24b-915ea70563c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:22.719720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6654b3-6537-49a2-b24b-915ea70563c8 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:22.730269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1279/2544] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:22.730590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6654b3-6537-49a2-b24b-915ea70563c8 service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:22.780966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7d989-5e6e-4979-b5ac-61afb1135bf1 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:22.786511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac40d67-6105-43d1-8206-bc3eb562474e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:22.787318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac40d67-6105-43d1-8206-bc3eb562474e admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.787709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac40d67-6105-43d1-8206-bc3eb562474e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.788093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac40d67-6105-43d1-8206-bc3eb562474e admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.788869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac40d67-6105-43d1-8206-bc3eb562474e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:22.790870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac40d67-6105-43d1-8206-bc3eb562474e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:22.799313 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6654b3-6537-49a2-b24b-915ea70563c8 service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:22.806119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a3e56-1fbb-4c64-b080-0a9253e3bf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:22.806793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a3e56-1fbb-4c64-b080-0a9253e3bf72 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.807076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a3e56-1fbb-4c64-b080-0a9253e3bf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.807351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a3e56-1fbb-4c64-b080-0a9253e3bf72 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.808047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a3e56-1fbb-4c64-b080-0a9253e3bf72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:22.810014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a3e56-1fbb-4c64-b080-0a9253e3bf72 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:22.878489 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:22.917636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1266/2545] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:22 2022] POST /identity/v3/users => generated 295 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:22.919762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac40d67-6105-43d1-8206-bc3eb562474e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:22.928287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac40d67-6105-43d1-8206-bc3eb562474e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:22.932894 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484689cd-62eb-48c5-afc2-d758aa7fa513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:22.933565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484689cd-62eb-48c5-afc2-d758aa7fa513 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:22.933849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484689cd-62eb-48c5-afc2-d758aa7fa513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:22.934137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484689cd-62eb-48c5-afc2-d758aa7fa513 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:22.934807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484689cd-62eb-48c5-afc2-d758aa7fa513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:22.936719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484689cd-62eb-48c5-afc2-d758aa7fa513 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:22.986770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1280/2546] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:22 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:22.987693 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a3e56-1fbb-4c64-b080-0a9253e3bf72 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:22.997682 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a3e56-1fbb-4c64-b080-0a9253e3bf72 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.007057 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f289-b50e-4fdf-82fd-84c4ba58de39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.008925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f289-b50e-4fdf-82fd-84c4ba58de39 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.017499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f289-b50e-4fdf-82fd-84c4ba58de39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.017499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f289-b50e-4fdf-82fd-84c4ba58de39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.017499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2f289-b50e-4fdf-82fd-84c4ba58de39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.017499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2f289-b50e-4fdf-82fd-84c4ba58de39 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.023307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1281/2547] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:23.024319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f289-b50e-4fdf-82fd-84c4ba58de39 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.034751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f289-b50e-4fdf-82fd-84c4ba58de39 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.039462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aee8d4-a599-4bbc-b035-83861ecedca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.040119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aee8d4-a599-4bbc-b035-83861ecedca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.040403 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aee8d4-a599-4bbc-b035-83861ecedca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.040680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aee8d4-a599-4bbc-b035-83861ecedca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.041440 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aee8d4-a599-4bbc-b035-83861ecedca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.043022 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aee8d4-a599-4bbc-b035-83861ecedca9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.053807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1267/2548] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:22 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:23.063372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484689cd-62eb-48c5-afc2-d758aa7fa513 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.063541 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1282/2549] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:23.064539 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484689cd-62eb-48c5-afc2-d758aa7fa513 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.066677 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aee8d4-a599-4bbc-b035-83861ecedca9 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.071099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae64b9b-5707-402e-a31c-de72bcadf0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.072013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae64b9b-5707-402e-a31c-de72bcadf0fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.072304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae64b9b-5707-402e-a31c-de72bcadf0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.072584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae64b9b-5707-402e-a31c-de72bcadf0fc admin admin] PATH_INFO: `/v3/projects/d950a314f32e441c8ad7d87b8e748507/users/8d18d4ac05c245e88c3a57846065125b/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.077508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aee8d4-a599-4bbc-b035-83861ecedca9 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.082254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c326c-c596-47c0-b117-2a1f9bff870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.082925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c326c-c596-47c0-b117-2a1f9bff870c admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.083211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c326c-c596-47c0-b117-2a1f9bff870c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.083485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c326c-c596-47c0-b117-2a1f9bff870c admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.084219 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c326c-c596-47c0-b117-2a1f9bff870c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.085867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c326c-c596-47c0-b117-2a1f9bff870c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.089805 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae64b9b-5707-402e-a31c-de72bcadf0fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d950a314f32e441c8ad7d87b8e748507, user_id=8d18d4ac05c245e88c3a57846065125b, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.092622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae64b9b-5707-402e-a31c-de72bcadf0fc admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.096881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1283/2550] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:51:23.097820 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c326c-c596-47c0-b117-2a1f9bff870c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.107638 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c326c-c596-47c0-b117-2a1f9bff870c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.112867 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44a033-af88-4217-bc71-0508ffaadd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.114829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44a033-af88-4217-bc71-0508ffaadd58 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.114829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44a033-af88-4217-bc71-0508ffaadd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.114829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44a033-af88-4217-bc71-0508ffaadd58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.114829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44a033-af88-4217-bc71-0508ffaadd58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.116379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44a033-af88-4217-bc71-0508ffaadd58 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.126134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1284/2551] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:23.127571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44a033-af88-4217-bc71-0508ffaadd58 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.137184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44a033-af88-4217-bc71-0508ffaadd58 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.141888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2335db6-6fca-42aa-87ef-55d7b77bf134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.143316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2335db6-6fca-42aa-87ef-55d7b77bf134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.143601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2335db6-6fca-42aa-87ef-55d7b77bf134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.143874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2335db6-6fca-42aa-87ef-55d7b77bf134 admin admin] PATH_INFO: `/v3/projects/eddd674a53754221ba3a333c0c065d1b/users/eb5a667fa9c24fda8cdb8e99e74066bd/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.158781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2335db6-6fca-42aa-87ef-55d7b77bf134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddd674a53754221ba3a333c0c065d1b, user_id=eb5a667fa9c24fda8cdb8e99e74066bd, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.161498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2335db6-6fca-42aa-87ef-55d7b77bf134 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.208151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1268/2552] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:23 2022] PUT /identity/v3/projects/d950a314f32e441c8ad7d87b8e748507/users/8d18d4ac05c245e88c3a57846065125b/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:23.210344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae64b9b-5707-402e-a31c-de72bcadf0fc admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.237123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1285/2553] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:23 2022] PUT /identity/v3/projects/eddd674a53754221ba3a333c0c065d1b/users/eb5a667fa9c24fda8cdb8e99e74066bd/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:23.238686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2335db6-6fca-42aa-87ef-55d7b77bf134 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.273107 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae64b9b-5707-402e-a31c-de72bcadf0fc admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.309613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2335db6-6fca-42aa-87ef-55d7b77bf134 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.315751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63251ba1-060b-4a5e-a0ee-9ca14761d37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.316435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251ba1-060b-4a5e-a0ee-9ca14761d37b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.316720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251ba1-060b-4a5e-a0ee-9ca14761d37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.316998 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251ba1-060b-4a5e-a0ee-9ca14761d37b admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.317775 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63251ba1-060b-4a5e-a0ee-9ca14761d37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.319423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63251ba1-060b-4a5e-a0ee-9ca14761d37b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.332881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1286/2554] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:23.332881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63251ba1-060b-4a5e-a0ee-9ca14761d37b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.340945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7a370-f61e-45fb-9e6b-d4b4bf734924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.342109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7a370-f61e-45fb-9e6b-d4b4bf734924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.342258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7a370-f61e-45fb-9e6b-d4b4bf734924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.342591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7a370-f61e-45fb-9e6b-d4b4bf734924 admin admin] PATH_INFO: `/v3/projects/7119e46a06b54c2984b21f32d9c156e3/users/1a71cad9c9114c5cb0bf8b4fe982e4d5/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.342876 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63251ba1-060b-4a5e-a0ee-9ca14761d37b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.349211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c681-34ba-4fcd-8924-48ee93ec72ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.349959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c681-34ba-4fcd-8924-48ee93ec72ac admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.350245 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c681-34ba-4fcd-8924-48ee93ec72ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.350530 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c681-34ba-4fcd-8924-48ee93ec72ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.351266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c681-34ba-4fcd-8924-48ee93ec72ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.352923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c681-34ba-4fcd-8924-48ee93ec72ac admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.358506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7a370-f61e-45fb-9e6b-d4b4bf734924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7119e46a06b54c2984b21f32d9c156e3, user_id=1a71cad9c9114c5cb0bf8b4fe982e4d5, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.361343 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7a370-f61e-45fb-9e6b-d4b4bf734924 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.366935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1287/2555] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:23.366935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9658bc-f78c-4fac-a7e0-7532b01714a5 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.366935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9658bc-f78c-4fac-a7e0-7532b01714a5 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.366935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9658bc-f78c-4fac-a7e0-7532b01714a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.395180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-eb9658bc-f78c-4fac-a7e0-7532b01714a5 None None] MFA Rules not processed for user `4a87961bce7e4333bbe1dc29dfce9617`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:23.397224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eb9658bc-f78c-4fac-a7e0-7532b01714a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:23.463305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1269/2556] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:23 2022] PUT /identity/v3/projects/7119e46a06b54c2984b21f32d9c156e3/users/1a71cad9c9114c5cb0bf8b4fe982e4d5/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:23.468419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7a370-f61e-45fb-9e6b-d4b4bf734924 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.469982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1288/2557] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:23 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:23.471104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9658bc-f78c-4fac-a7e0-7532b01714a5 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.482409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9658bc-f78c-4fac-a7e0-7532b01714a5 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.527706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7a370-f61e-45fb-9e6b-d4b4bf734924 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.538332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cebfad-d6df-4c71-8f2b-39b6abd5455d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.539332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cebfad-d6df-4c71-8f2b-39b6abd5455d admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.539616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cebfad-d6df-4c71-8f2b-39b6abd5455d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.539891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cebfad-d6df-4c71-8f2b-39b6abd5455d admin admin] PATH_INFO: `/v3/projects/d950a314f32e441c8ad7d87b8e748507/users/8d18d4ac05c245e88c3a57846065125b/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.542129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cebfad-d6df-4c71-8f2b-39b6abd5455d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d950a314f32e441c8ad7d87b8e748507, user_id=8d18d4ac05c245e88c3a57846065125b, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.543229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4099503-5460-4bf9-8c44-96cd12218c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.544296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4099503-5460-4bf9-8c44-96cd12218c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.544583 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4099503-5460-4bf9-8c44-96cd12218c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.544836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4099503-5460-4bf9-8c44-96cd12218c23 admin admin] PATH_INFO: `/v3/projects/eddd674a53754221ba3a333c0c065d1b/users/eb5a667fa9c24fda8cdb8e99e74066bd/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.545199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cebfad-d6df-4c71-8f2b-39b6abd5455d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.547317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4099503-5460-4bf9-8c44-96cd12218c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddd674a53754221ba3a333c0c065d1b, user_id=eb5a667fa9c24fda8cdb8e99e74066bd, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.550564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4099503-5460-4bf9-8c44-96cd12218c23 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.558780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1270/2558] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:23 2022] PUT /identity/v3/projects/d950a314f32e441c8ad7d87b8e748507/users/8d18d4ac05c245e88c3a57846065125b/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:23.560252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1b4d4-b6e0-4c92-98de-22617136842f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.560617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1b4d4-b6e0-4c92-98de-22617136842f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.560825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1b4d4-b6e0-4c92-98de-22617136842f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.563042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1289/2559] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:23 2022] PUT /identity/v3/projects/eddd674a53754221ba3a333c0c065d1b/users/eb5a667fa9c24fda8cdb8e99e74066bd/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:23.566327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4099503-5460-4bf9-8c44-96cd12218c23 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.570469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d4099503-5460-4bf9-8c44-96cd12218c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:23.580454 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-1ad1b4d4-b6e0-4c92-98de-22617136842f None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:23.581796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1ad1b4d4-b6e0-4c92-98de-22617136842f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:23.650707 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4099503-5460-4bf9-8c44-96cd12218c23 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.653553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1271/2560] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:23.654947 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614315-516d-44c3-8313-29f2c7ddb9cb None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.655241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614315-516d-44c3-8313-29f2c7ddb9cb None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.655516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614315-516d-44c3-8313-29f2c7ddb9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.659705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83416e12-bdb1-45ba-9c65-16b8670b4122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.660825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83416e12-bdb1-45ba-9c65-16b8670b4122 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.661630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83416e12-bdb1-45ba-9c65-16b8670b4122 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.662088 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83416e12-bdb1-45ba-9c65-16b8670b4122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.664885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-83416e12-bdb1-45ba-9c65-16b8670b4122 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:23.688283 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9b614315-516d-44c3-8313-29f2c7ddb9cb None None] MFA Rules not processed for user `eb5a667fa9c24fda8cdb8e99e74066bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:23.690271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9b614315-516d-44c3-8313-29f2c7ddb9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:23.728666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83416e12-bdb1-45ba-9c65-16b8670b4122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.730314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83416e12-bdb1-45ba-9c65-16b8670b4122 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.739848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1290/2561] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:23.741281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc5134-752e-41e7-aaf2-b421dba9d3e1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.741568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc5134-752e-41e7-aaf2-b421dba9d3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.741853 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc5134-752e-41e7-aaf2-b421dba9d3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.762482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1272/2562] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:23.763340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614315-516d-44c3-8313-29f2c7ddb9cb None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.766723 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9b614315-516d-44c3-8313-29f2c7ddb9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:23.772004 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-18bc5134-752e-41e7-aaf2-b421dba9d3e1 None None] MFA Rules not processed for user `8d18d4ac05c245e88c3a57846065125b`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:23.774524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-18bc5134-752e-41e7-aaf2-b421dba9d3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:23.843260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614315-516d-44c3-8313-29f2c7ddb9cb None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.848647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115237f4-f5a0-4e12-9939-8d3921ca9fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.849305 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115237f4-f5a0-4e12-9939-8d3921ca9fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.849587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115237f4-f5a0-4e12-9939-8d3921ca9fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.849862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115237f4-f5a0-4e12-9939-8d3921ca9fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.860027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115237f4-f5a0-4e12-9939-8d3921ca9fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.867111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115237f4-f5a0-4e12-9939-8d3921ca9fb8 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.867310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1291/2563] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:23.867310 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc5134-752e-41e7-aaf2-b421dba9d3e1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:23.868281 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-18bc5134-752e-41e7-aaf2-b421dba9d3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:23.876025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1273/2564] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:23.886123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc5134-752e-41e7-aaf2-b421dba9d3e1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:23.888801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3367eac-3fa3-4cd1-acfa-86b1d5af4fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:23.889489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3367eac-3fa3-4cd1-acfa-86b1d5af4fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:23.889771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3367eac-3fa3-4cd1-acfa-86b1d5af4fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:23.890048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3367eac-3fa3-4cd1-acfa-86b1d5af4fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:23.928160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:23.956042 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3367eac-3fa3-4cd1-acfa-86b1d5af4fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:23.957507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3367eac-3fa3-4cd1-acfa-86b1d5af4fd1 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:23.967754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1292/2565] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:24.036181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115237f4-f5a0-4e12-9939-8d3921ca9fb8 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:24.045816 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115237f4-f5a0-4e12-9939-8d3921ca9fb8 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:24.052081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534c246-dd99-4b27-ab79-0cff6f664e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:24.052081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534c246-dd99-4b27-ab79-0cff6f664e9d service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:24.052081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534c246-dd99-4b27-ab79-0cff6f664e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:24.052081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534c246-dd99-4b27-ab79-0cff6f664e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:24.113133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534c246-dd99-4b27-ab79-0cff6f664e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:24.114626 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534c246-dd99-4b27-ab79-0cff6f664e9d service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:24.123906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1274/2566] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:25.354675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:25.986444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3367eac-3fa3-4cd1-acfa-86b1d5af4fd1 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:25.997182 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3367eac-3fa3-4cd1-acfa-86b1d5af4fd1 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:26.001549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b3365-6daa-45fb-953e-53d186ab6fb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:26.002280 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b3365-6daa-45fb-953e-53d186ab6fb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:26.002659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b3365-6daa-45fb-953e-53d186ab6fb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:26.002981 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b3365-6daa-45fb-953e-53d186ab6fb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:26.085194 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b3365-6daa-45fb-953e-53d186ab6fb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:26.087060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b3365-6daa-45fb-953e-53d186ab6fb3 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:26.098606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1293/2567] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:26.449364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:26.663373 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534c246-dd99-4b27-ab79-0cff6f664e9d service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:26.678676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534c246-dd99-4b27-ab79-0cff6f664e9d service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:26.683940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:26.685526 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:26.685826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:26.686099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] PATH_INFO: `/v3/users/0db86d876f484602abe6964ea79c8da8` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:26.687455 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db86d876f484602abe6964ea79c8da8)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:26.689921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:26.948511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db86d876f484602abe6964ea79c8da8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:26.949749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db86d876f484602abe6964ea79c8da8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:26.968518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db86d876f484602abe6964ea79c8da8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:26.982484 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db86d876f484602abe6964ea79c8da8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:27.071605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db86d876f484602abe6964ea79c8da8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:27.084542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1275/2568] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:26 2022] DELETE /identity/v3/users/0db86d876f484602abe6964ea79c8da8 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:27.089487 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a4fd4-c6c0-48fe-9079-ec2150a24794 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:27.090119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a4fd4-c6c0-48fe-9079-ec2150a24794 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:27.090585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a4fd4-c6c0-48fe-9079-ec2150a24794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:27.117897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-238a4fd4-c6c0-48fe-9079-ec2150a24794 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:27.119488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-238a4fd4-c6c0-48fe-9079-ec2150a24794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:27.199030 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1294/2569] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:27.211664 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:27.213280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:27.283984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d254-6a2a-473e-b52b-538dbc0f2d17 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:27.289684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a0723-9c82-47ec-bbf8-dd51ac1fb525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:27.290351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a0723-9c82-47ec-bbf8-dd51ac1fb525 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:27.290636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a0723-9c82-47ec-bbf8-dd51ac1fb525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:27.290980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a0723-9c82-47ec-bbf8-dd51ac1fb525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:27.302756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a0723-9c82-47ec-bbf8-dd51ac1fb525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:27.304416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a0723-9c82-47ec-bbf8-dd51ac1fb525 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:27.314834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1276/2570] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:27.788752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:28.046835 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a4fd4-c6c0-48fe-9079-ec2150a24794 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:28.049831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-238a4fd4-c6c0-48fe-9079-ec2150a24794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:28.065549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a4fd4-c6c0-48fe-9079-ec2150a24794 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:28.071368 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:28.072348 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:28.072698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:28.073373 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] PATH_INFO: `/v3/projects/0f68263b01c64cbd8c759ba840c949f7` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:28.074692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f68263b01c64cbd8c759ba840c949f7)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:28.077635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:28.109731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:28.189724 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f68263b01c64cbd8c759ba840c949f7', 'request_id': 'req-94a47fbb-4426-4558-8781-ae48f816acaa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:28.263560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f68263b01c64cbd8c759ba840c949f7', 'request_id': 'req-94a47fbb-4426-4558-8781-ae48f816acaa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:28.274369 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f68263b01c64cbd8c759ba840c949f7', 'request_id': 'req-94a47fbb-4426-4558-8781-ae48f816acaa'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:28.288643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] The token cache is being invalidate because project 0f68263b01c64cbd8c759ba840c949f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:28.289056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:28.289857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-94a47fbb-4426-4558-8781-ae48f816acaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:28.291752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1295/2571] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:28 2022] DELETE /identity/v3/projects/0f68263b01c64cbd8c759ba840c949f7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:28.297318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd17d-2cef-4c58-a9df-c5ff115efcaa None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:28.297940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd17d-2cef-4c58-a9df-c5ff115efcaa None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:28.298310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd17d-2cef-4c58-a9df-c5ff115efcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:28.320473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-229dd17d-2cef-4c58-a9df-c5ff115efcaa None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:28.321900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-229dd17d-2cef-4c58-a9df-c5ff115efcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:28.421610 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1277/2572] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:28.426548 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa7bc9-d508-41f2-aa02-9619087c99f3 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:28.426974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa7bc9-d508-41f2-aa02-9619087c99f3 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:28.428029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa7bc9-d508-41f2-aa02-9619087c99f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:28.457294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c8fa7bc9-d508-41f2-aa02-9619087c99f3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:28.461696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c8fa7bc9-d508-41f2-aa02-9619087c99f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:28.477624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1296/2573] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:28.478676 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dd17d-2cef-4c58-a9df-c5ff115efcaa None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:28.493421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dd17d-2cef-4c58-a9df-c5ff115efcaa None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:28.498913 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12f1cf-49a3-4842-9e20-4edf7aa0ded3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:28.499606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12f1cf-49a3-4842-9e20-4edf7aa0ded3 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:28.500057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12f1cf-49a3-4842-9e20-4edf7aa0ded3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:28.500346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12f1cf-49a3-4842-9e20-4edf7aa0ded3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:28.501098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12f1cf-49a3-4842-9e20-4edf7aa0ded3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:28.501467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12f1cf-49a3-4842-9e20-4edf7aa0ded3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:28.504799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12f1cf-49a3-4842-9e20-4edf7aa0ded3 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:28.520544 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1278/2574] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:51:28.524414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa7bc9-d508-41f2-aa02-9619087c99f3 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:28.543142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa7bc9-d508-41f2-aa02-9619087c99f3 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:28.547992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73e8d7-3c63-4e0e-b84e-a5ea7c078fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:28.549264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73e8d7-3c63-4e0e-b84e-a5ea7c078fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:28.549545 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73e8d7-3c63-4e0e-b84e-a5ea7c078fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:28.549812 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73e8d7-3c63-4e0e-b84e-a5ea7c078fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:28.550520 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73e8d7-3c63-4e0e-b84e-a5ea7c078fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:28.552426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73e8d7-3c63-4e0e-b84e-a5ea7c078fd1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:28.668589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12f1cf-49a3-4842-9e20-4edf7aa0ded3 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:28.676669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1297/2575] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:28 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:28.680557 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12f1cf-49a3-4842-9e20-4edf7aa0ded3 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:28.748410 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595eba4-cf38-4963-8926-ae278a7e1173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:28.749573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595eba4-cf38-4963-8926-ae278a7e1173 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:28.750013 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595eba4-cf38-4963-8926-ae278a7e1173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:28.750443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595eba4-cf38-4963-8926-ae278a7e1173 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:28.751302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595eba4-cf38-4963-8926-ae278a7e1173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:28.753820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595eba4-cf38-4963-8926-ae278a7e1173 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:28.864234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1279/2576] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:28 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:28.869024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73e8d7-3c63-4e0e-b84e-a5ea7c078fd1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:28.879255 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73e8d7-3c63-4e0e-b84e-a5ea7c078fd1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:28.884256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07648c-808c-4a1d-94b1-0f76201a9f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:28.884990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07648c-808c-4a1d-94b1-0f76201a9f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:28.885330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07648c-808c-4a1d-94b1-0f76201a9f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:28.885587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07648c-808c-4a1d-94b1-0f76201a9f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:28.886314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07648c-808c-4a1d-94b1-0f76201a9f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:28.887932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07648c-808c-4a1d-94b1-0f76201a9f87 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:28.899924 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1298/2577] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:28.903582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595eba4-cf38-4963-8926-ae278a7e1173 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:28.917235 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595eba4-cf38-4963-8926-ae278a7e1173 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:28.924724 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e08b7-d0b8-4de2-88d5-bb45a90c4d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:28.925505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e08b7-d0b8-4de2-88d5-bb45a90c4d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:28.925792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e08b7-d0b8-4de2-88d5-bb45a90c4d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:28.926065 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e08b7-d0b8-4de2-88d5-bb45a90c4d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:28.926878 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e08b7-d0b8-4de2-88d5-bb45a90c4d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:28.928407 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e08b7-d0b8-4de2-88d5-bb45a90c4d51 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:28.942154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1280/2578] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:51:28.943721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07648c-808c-4a1d-94b1-0f76201a9f87 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:28.959174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07648c-808c-4a1d-94b1-0f76201a9f87 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:28.966760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8549a11-1df7-4e26-a9d0-bb1111b45286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:28.967508 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8549a11-1df7-4e26-a9d0-bb1111b45286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:28.967790 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8549a11-1df7-4e26-a9d0-bb1111b45286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:28.968072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8549a11-1df7-4e26-a9d0-bb1111b45286 admin admin] PATH_INFO: `/v3/projects/e23b064dd6fd481a97cf3a9c47cb98db/users/d61090e529984229aa0f6841a4bb122b/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:28.998735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8549a11-1df7-4e26-a9d0-bb1111b45286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23b064dd6fd481a97cf3a9c47cb98db, user_id=d61090e529984229aa0f6841a4bb122b, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:29.001517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8549a11-1df7-4e26-a9d0-bb1111b45286 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:29.103613 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1299/2579] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:28 2022] PUT /identity/v3/projects/e23b064dd6fd481a97cf3a9c47cb98db/users/d61090e529984229aa0f6841a4bb122b/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:29.105780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7960a40-fd04-4f24-b110-e57a497b736c None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:29.107330 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7960a40-fd04-4f24-b110-e57a497b736c None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:29.107809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7960a40-fd04-4f24-b110-e57a497b736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:29.145036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-a7960a40-fd04-4f24-b110-e57a497b736c None None] MFA Rules not processed for user `d61090e529984229aa0f6841a4bb122b`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:29.146682 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a7960a40-fd04-4f24-b110-e57a497b736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:29.229924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1281/2580] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:29.241190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8549a11-1df7-4e26-a9d0-bb1111b45286 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:29.242992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b8549a11-1df7-4e26-a9d0-bb1111b45286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:29.421957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8549a11-1df7-4e26-a9d0-bb1111b45286 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:29.430349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da5397-9e5b-44ca-b603-0b692ba7082c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:29.431483 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da5397-9e5b-44ca-b603-0b692ba7082c service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:29.432681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da5397-9e5b-44ca-b603-0b692ba7082c service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:29.433249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da5397-9e5b-44ca-b603-0b692ba7082c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:29.447083 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da5397-9e5b-44ca-b603-0b692ba7082c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:29.448959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da5397-9e5b-44ca-b603-0b692ba7082c service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:29.463022 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1300/2581] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:29.773756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7960a40-fd04-4f24-b110-e57a497b736c None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:29.856183 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da5397-9e5b-44ca-b603-0b692ba7082c service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:29.857833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-61da5397-9e5b-44ca-b603-0b692ba7082c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:29.884438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7960a40-fd04-4f24-b110-e57a497b736c None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:29.895140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:29.896819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:29.897388 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:29.897940 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] PATH_INFO: `/v3/users/d61090e529984229aa0f6841a4bb122b` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:29.900018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d61090e529984229aa0f6841a4bb122b)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:29.905773 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:29.957930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:29.969463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da5397-9e5b-44ca-b603-0b692ba7082c service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:29.977533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd345c6f-8880-4d0c-8d44-837ed4d71d7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:29.980438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd345c6f-8880-4d0c-8d44-837ed4d71d7a service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:29.980438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd345c6f-8880-4d0c-8d44-837ed4d71d7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:29.980438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd345c6f-8880-4d0c-8d44-837ed4d71d7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:29.983366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bd345c6f-8880-4d0c-8d44-837ed4d71d7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:30.091200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd345c6f-8880-4d0c-8d44-837ed4d71d7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:30.093094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd345c6f-8880-4d0c-8d44-837ed4d71d7a service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:30.118343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1301/2582] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:30.176574 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd61090e529984229aa0f6841a4bb122b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:30.177655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd61090e529984229aa0f6841a4bb122b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:30.211243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd61090e529984229aa0f6841a4bb122b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:30.220427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd61090e529984229aa0f6841a4bb122b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:30.279147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64819fbe-1b35-4a6f-a27b-8bab970ba951 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:30.279604 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64819fbe-1b35-4a6f-a27b-8bab970ba951 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:30.279779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64819fbe-1b35-4a6f-a27b-8bab970ba951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:30.295090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-1b54e793-0d41-42a2-b482-b06972425ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd61090e529984229aa0f6841a4bb122b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:30.306646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1282/2583] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:29 2022] DELETE /identity/v3/users/d61090e529984229aa0f6841a4bb122b => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:30.308205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb761-a27e-49bb-bc14-a8cffd45badf None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:30.308496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb761-a27e-49bb-bc14-a8cffd45badf None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:30.308776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb761-a27e-49bb-bc14-a8cffd45badf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:30.312787 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-64819fbe-1b35-4a6f-a27b-8bab970ba951 None None] MFA Rules not processed for user `1a71cad9c9114c5cb0bf8b4fe982e4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:30.316975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-64819fbe-1b35-4a6f-a27b-8bab970ba951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:30.328835 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-328fb761-a27e-49bb-bc14-a8cffd45badf None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:30.330081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-328fb761-a27e-49bb-bc14-a8cffd45badf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:30.387237 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:30.407335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1283/2584] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:30.413582 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1302/2585] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:30.417765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fb761-a27e-49bb-bc14-a8cffd45badf None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:30.418612 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64819fbe-1b35-4a6f-a27b-8bab970ba951 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:30.420999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-328fb761-a27e-49bb-bc14-a8cffd45badf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:30.421405 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-64819fbe-1b35-4a6f-a27b-8bab970ba951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:30.431934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fb761-a27e-49bb-bc14-a8cffd45badf None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:30.437204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a6d30-986e-4c8c-91fe-531b4ad5a1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:30.437873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a6d30-986e-4c8c-91fe-531b4ad5a1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:30.438154 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a6d30-986e-4c8c-91fe-531b4ad5a1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:30.438426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a6d30-986e-4c8c-91fe-531b4ad5a1c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:30.439883 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a6d30-986e-4c8c-91fe-531b4ad5a1c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:30.442070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a6d30-986e-4c8c-91fe-531b4ad5a1c8 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:30.488515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64819fbe-1b35-4a6f-a27b-8bab970ba951 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:30.492763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfc161-14bc-4080-a974-f201907a8b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:30.495886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfc161-14bc-4080-a974-f201907a8b8c service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:30.495886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfc161-14bc-4080-a974-f201907a8b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:30.495886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfc161-14bc-4080-a974-f201907a8b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:30.505466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfc161-14bc-4080-a974-f201907a8b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:30.506904 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfc161-14bc-4080-a974-f201907a8b8c service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:30.517579 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1303/2586] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:30.580757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1284/2587] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:30 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:30.582538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfc161-14bc-4080-a974-f201907a8b8c service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:30.602386 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfc161-14bc-4080-a974-f201907a8b8c service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:30.688158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3096008-87f4-43aa-adc9-cc8d11cf7040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:30.689230 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3096008-87f4-43aa-adc9-cc8d11cf7040 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:30.689383 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3096008-87f4-43aa-adc9-cc8d11cf7040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:30.689624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3096008-87f4-43aa-adc9-cc8d11cf7040 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:30.690338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3096008-87f4-43aa-adc9-cc8d11cf7040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:30.692648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3096008-87f4-43aa-adc9-cc8d11cf7040 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:30.838731 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1304/2588] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:30 2022] POST /identity/v3/users => generated 296 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:30.843055 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a6d30-986e-4c8c-91fe-531b4ad5a1c8 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:30.855394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a6d30-986e-4c8c-91fe-531b4ad5a1c8 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:30.861494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1c471-1ee1-4508-bb64-3e5dbc1ffa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:30.865346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1c471-1ee1-4508-bb64-3e5dbc1ffa14 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:30.865346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1c471-1ee1-4508-bb64-3e5dbc1ffa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:30.865346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1c471-1ee1-4508-bb64-3e5dbc1ffa14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:30.865346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1c471-1ee1-4508-bb64-3e5dbc1ffa14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:30.866850 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1c471-1ee1-4508-bb64-3e5dbc1ffa14 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:30.880939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1285/2589] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:30.884839 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3096008-87f4-43aa-adc9-cc8d11cf7040 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:30.895121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3096008-87f4-43aa-adc9-cc8d11cf7040 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:30.900555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8432b-e9a0-4d26-bcf4-9f8bd98bffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:30.901524 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8432b-e9a0-4d26-bcf4-9f8bd98bffec admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:30.901814 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8432b-e9a0-4d26-bcf4-9f8bd98bffec admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:30.902092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8432b-e9a0-4d26-bcf4-9f8bd98bffec admin admin] PATH_INFO: `/v3/projects/bc8c54170e7b45399e6ff7218f53ab8e/users/d2f8dbd621a74ea29ff3621a49a5de32/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:30.921754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8432b-e9a0-4d26-bcf4-9f8bd98bffec admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8c54170e7b45399e6ff7218f53ab8e, user_id=d2f8dbd621a74ea29ff3621a49a5de32, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:30.925038 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8432b-e9a0-4d26-bcf4-9f8bd98bffec admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:30.976911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1c471-1ee1-4508-bb64-3e5dbc1ffa14 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:30.978810 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3ed1c471-1ee1-4508-bb64-3e5dbc1ffa14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:31.002439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1305/2590] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:30 2022] PUT /identity/v3/projects/bc8c54170e7b45399e6ff7218f53ab8e/users/d2f8dbd621a74ea29ff3621a49a5de32/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:31.005555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8432b-e9a0-4d26-bcf4-9f8bd98bffec admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:31.011155 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1c471-1ee1-4508-bb64-3e5dbc1ffa14 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:31.110023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8432b-e9a0-4d26-bcf4-9f8bd98bffec admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:31.121747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91eab19-cb73-4ceb-8fa0-4c0268728e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:31.122488 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91eab19-cb73-4ceb-8fa0-4c0268728e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:31.123021 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91eab19-cb73-4ceb-8fa0-4c0268728e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:31.123538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91eab19-cb73-4ceb-8fa0-4c0268728e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:31.124529 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91eab19-cb73-4ceb-8fa0-4c0268728e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:31.126793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91eab19-cb73-4ceb-8fa0-4c0268728e0c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:31.142948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1306/2591] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:31.148023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91eab19-cb73-4ceb-8fa0-4c0268728e0c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:31.151376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:31.152376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:31.155781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:31.155781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] PATH_INFO: `/v3/projects/e23b064dd6fd481a97cf3a9c47cb98db` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:31.158494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e23b064dd6fd481a97cf3a9c47cb98db)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:31.160714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:31.167128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91eab19-cb73-4ceb-8fa0-4c0268728e0c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:31.174462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab13748-89ff-4bac-8243-e5ccb5229f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:31.175557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab13748-89ff-4bac-8243-e5ccb5229f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:31.175903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab13748-89ff-4bac-8243-e5ccb5229f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:31.176291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab13748-89ff-4bac-8243-e5ccb5229f65 admin admin] PATH_INFO: `/v3/projects/bc8c54170e7b45399e6ff7218f53ab8e/users/d2f8dbd621a74ea29ff3621a49a5de32/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:31.178451 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab13748-89ff-4bac-8243-e5ccb5229f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8c54170e7b45399e6ff7218f53ab8e, user_id=d2f8dbd621a74ea29ff3621a49a5de32, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:31.181328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab13748-89ff-4bac-8243-e5ccb5229f65 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:31.198656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1307/2592] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:31 2022] PUT /identity/v3/projects/bc8c54170e7b45399e6ff7218f53ab8e/users/d2f8dbd621a74ea29ff3621a49a5de32/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:31.273375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e23b064dd6fd481a97cf3a9c47cb98db', 'request_id': 'req-e60649c8-a21a-401f-95e6-080073f3fa5c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:31.280167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e23b064dd6fd481a97cf3a9c47cb98db', 'request_id': 'req-e60649c8-a21a-401f-95e6-080073f3fa5c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:31.286697 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e23b064dd6fd481a97cf3a9c47cb98db', 'request_id': 'req-e60649c8-a21a-401f-95e6-080073f3fa5c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:31.389609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] The token cache is being invalidate because project e23b064dd6fd481a97cf3a9c47cb98db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:31.390145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:31.392020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-e60649c8-a21a-401f-95e6-080073f3fa5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:31.394580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1286/2593] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:30 2022] DELETE /identity/v3/projects/e23b064dd6fd481a97cf3a9c47cb98db => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:31.399673 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83005871-ef8d-4ec8-aed8-050214843726 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:31.401776 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83005871-ef8d-4ec8-aed8-050214843726 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:31.402067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83005871-ef8d-4ec8-aed8-050214843726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:31.423425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-83005871-ef8d-4ec8-aed8-050214843726 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:31.424710 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-83005871-ef8d-4ec8-aed8-050214843726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:31.496267 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1308/2594] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:31.507529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54352778-0297-4a80-81b5-08a30b76e7ac None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:31.508106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54352778-0297-4a80-81b5-08a30b76e7ac None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:31.508106 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54352778-0297-4a80-81b5-08a30b76e7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:31.531889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-54352778-0297-4a80-81b5-08a30b76e7ac None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:31.533933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-54352778-0297-4a80-81b5-08a30b76e7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:31.542840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1287/2595] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:51:31.547577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83005871-ef8d-4ec8-aed8-050214843726 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:31.558007 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83005871-ef8d-4ec8-aed8-050214843726 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:31.565001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d76261-7fda-41db-b1c2-a7295d24a266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:31.565737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d76261-7fda-41db-b1c2-a7295d24a266 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:31.566024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d76261-7fda-41db-b1c2-a7295d24a266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:31.566330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d76261-7fda-41db-b1c2-a7295d24a266 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:31.567139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d76261-7fda-41db-b1c2-a7295d24a266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:31.567847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d76261-7fda-41db-b1c2-a7295d24a266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:31.569643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d76261-7fda-41db-b1c2-a7295d24a266 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:31.587204 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1309/2596] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:51:31.588398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54352778-0297-4a80-81b5-08a30b76e7ac None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:31.598554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54352778-0297-4a80-81b5-08a30b76e7ac None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:31.607737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68c0f0-14aa-435e-814d-fb00c9559539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:31.607737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c0f0-14aa-435e-814d-fb00c9559539 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:31.607737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c0f0-14aa-435e-814d-fb00c9559539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:31.607737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c0f0-14aa-435e-814d-fb00c9559539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:31.608376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68c0f0-14aa-435e-814d-fb00c9559539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:31.608376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68c0f0-14aa-435e-814d-fb00c9559539 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:31.700595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1288/2597] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:31 2022] POST /identity/v3/projects => generated 405 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:31.703650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d76261-7fda-41db-b1c2-a7295d24a266 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:31.713581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:31.718510 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d76261-7fda-41db-b1c2-a7295d24a266 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:31.789078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284a240-9e4d-4931-9a5b-6ed89b0ce66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:31.789976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284a240-9e4d-4931-9a5b-6ed89b0ce66f admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:31.790330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284a240-9e4d-4931-9a5b-6ed89b0ce66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:31.790678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284a240-9e4d-4931-9a5b-6ed89b0ce66f admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:31.791451 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284a240-9e4d-4931-9a5b-6ed89b0ce66f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:31.795202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284a240-9e4d-4931-9a5b-6ed89b0ce66f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:31.931048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1310/2598] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:31 2022] POST /identity/v3/users => generated 303 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:31.934976 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68c0f0-14aa-435e-814d-fb00c9559539 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:31.947556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68c0f0-14aa-435e-814d-fb00c9559539 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:31.952727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302886e-b012-4313-a0ac-28c30267efd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:31.953597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302886e-b012-4313-a0ac-28c30267efd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:31.953955 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302886e-b012-4313-a0ac-28c30267efd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:31.954453 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302886e-b012-4313-a0ac-28c30267efd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:31.955518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302886e-b012-4313-a0ac-28c30267efd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:31.957619 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302886e-b012-4313-a0ac-28c30267efd0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:31.971897 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1289/2599] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:31.976005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284a240-9e4d-4931-9a5b-6ed89b0ce66f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:31.987142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284a240-9e4d-4931-9a5b-6ed89b0ce66f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:31.991979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea84025-0851-4eae-841f-a99d6aa02716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:31.992662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea84025-0851-4eae-841f-a99d6aa02716 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:31.992947 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea84025-0851-4eae-841f-a99d6aa02716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:31.993247 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea84025-0851-4eae-841f-a99d6aa02716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:31.994012 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea84025-0851-4eae-841f-a99d6aa02716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:31.995602 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea84025-0851-4eae-841f-a99d6aa02716 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:32.007414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1311/2600] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:51:32.011570 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302886e-b012-4313-a0ac-28c30267efd0 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:32.022309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302886e-b012-4313-a0ac-28c30267efd0 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:32.027375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3246e75-964b-471e-b37b-02fbe36d6cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:32.028268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3246e75-964b-471e-b37b-02fbe36d6cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:32.028550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3246e75-964b-471e-b37b-02fbe36d6cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:32.028829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3246e75-964b-471e-b37b-02fbe36d6cc9 admin admin] PATH_INFO: `/v3/projects/ffdbf7f4c8014ce2b32d25ac7a88f5fb/users/f42c331b02dc4eb4b0c00cd6cd276496/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:32.045836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3246e75-964b-471e-b37b-02fbe36d6cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdbf7f4c8014ce2b32d25ac7a88f5fb, user_id=f42c331b02dc4eb4b0c00cd6cd276496, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:32.049456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3246e75-964b-471e-b37b-02fbe36d6cc9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:32.124349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1290/2601] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:32 2022] PUT /identity/v3/projects/ffdbf7f4c8014ce2b32d25ac7a88f5fb/users/f42c331b02dc4eb4b0c00cd6cd276496/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:32.129623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13959591-8bb8-461e-af43-675405a1edb0 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:32.129919 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13959591-8bb8-461e-af43-675405a1edb0 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:32.130202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13959591-8bb8-461e-af43-675405a1edb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:32.150777 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-13959591-8bb8-461e-af43-675405a1edb0 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:32.152375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-13959591-8bb8-461e-af43-675405a1edb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:32.249262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1312/2602] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:32.261295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3246e75-964b-471e-b37b-02fbe36d6cc9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:32.264365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f3246e75-964b-471e-b37b-02fbe36d6cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:32.340873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3246e75-964b-471e-b37b-02fbe36d6cc9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:32.346653 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81287192-ac04-4975-a4db-8467776ec3da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:32.347434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81287192-ac04-4975-a4db-8467776ec3da service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:32.347858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81287192-ac04-4975-a4db-8467776ec3da service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:32.348406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81287192-ac04-4975-a4db-8467776ec3da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:32.360214 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81287192-ac04-4975-a4db-8467776ec3da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:32.361889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81287192-ac04-4975-a4db-8467776ec3da service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:32.373379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1291/2603] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:32.700511 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:33.214934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:33.820980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35bc08-c6c0-4f8c-981c-fe7911f7b43d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:33.821465 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35bc08-c6c0-4f8c-981c-fe7911f7b43d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:33.822401 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35bc08-c6c0-4f8c-981c-fe7911f7b43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:33.854464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-bc35bc08-c6c0-4f8c-981c-fe7911f7b43d None None] MFA Rules not processed for user `f42c331b02dc4eb4b0c00cd6cd276496`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:33.856095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-bc35bc08-c6c0-4f8c-981c-fe7911f7b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:33.950626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1313/2604] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:33 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:33.955487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81287192-ac04-4975-a4db-8467776ec3da service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:33.968970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81287192-ac04-4975-a4db-8467776ec3da service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:33.977417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27c020-0aea-4f4b-a835-8fdc4a0f5fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:33.978081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27c020-0aea-4f4b-a835-8fdc4a0f5fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:33.978368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27c020-0aea-4f4b-a835-8fdc4a0f5fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:33.978647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27c020-0aea-4f4b-a835-8fdc4a0f5fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:33.979885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27c020-0aea-4f4b-a835-8fdc4a0f5fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:33.982627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27c020-0aea-4f4b-a835-8fdc4a0f5fcd admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:34.061977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1292/2605] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:33 2022] POST /identity/v3/projects => generated 405 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:34.065946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35bc08-c6c0-4f8c-981c-fe7911f7b43d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:34.080332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35bc08-c6c0-4f8c-981c-fe7911f7b43d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:34.182071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fcc10-7f99-4396-8a3f-b965b0f897bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:34.182685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fcc10-7f99-4396-8a3f-b965b0f897bf admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:34.182980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fcc10-7f99-4396-8a3f-b965b0f897bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:34.183263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fcc10-7f99-4396-8a3f-b965b0f897bf admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:34.184349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fcc10-7f99-4396-8a3f-b965b0f897bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:34.187556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fcc10-7f99-4396-8a3f-b965b0f897bf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:34.300282 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1314/2606] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:34 2022] POST /identity/v3/users => generated 309 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:34.304372 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27c020-0aea-4f4b-a835-8fdc4a0f5fcd admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:34.319571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27c020-0aea-4f4b-a835-8fdc4a0f5fcd admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:34.328054 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00ff41-be5b-407d-b27b-990e65909f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:34.328938 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00ff41-be5b-407d-b27b-990e65909f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:34.329430 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00ff41-be5b-407d-b27b-990e65909f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:34.330366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00ff41-be5b-407d-b27b-990e65909f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:34.331291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00ff41-be5b-407d-b27b-990e65909f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:34.333445 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00ff41-be5b-407d-b27b-990e65909f35 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:34.347666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1293/2607] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:34.352064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fcc10-7f99-4396-8a3f-b965b0f897bf admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:34.369431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fcc10-7f99-4396-8a3f-b965b0f897bf admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:34.375202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c012f-95c9-4269-8309-530ab53b8079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:34.377273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c012f-95c9-4269-8309-530ab53b8079 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:34.377595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c012f-95c9-4269-8309-530ab53b8079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:34.377980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c012f-95c9-4269-8309-530ab53b8079 admin admin] PATH_INFO: `/v3/projects/f6def5605649401486b1f35056fe5af2/users/4cca55f5dee24c0881dbe9f5db64f05e/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:34.397863 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c012f-95c9-4269-8309-530ab53b8079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6def5605649401486b1f35056fe5af2, user_id=4cca55f5dee24c0881dbe9f5db64f05e, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:34.401115 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c012f-95c9-4269-8309-530ab53b8079 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:34.548625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1315/2608] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:34 2022] PUT /identity/v3/projects/f6def5605649401486b1f35056fe5af2/users/4cca55f5dee24c0881dbe9f5db64f05e/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:34.551400 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00ff41-be5b-407d-b27b-990e65909f35 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:34.562635 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00ff41-be5b-407d-b27b-990e65909f35 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:34.656860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfef747-6906-40db-a39d-00b912bffab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:34.663606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfef747-6906-40db-a39d-00b912bffab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:34.664236 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfef747-6906-40db-a39d-00b912bffab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:34.664517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfef747-6906-40db-a39d-00b912bffab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:34.672662 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfef747-6906-40db-a39d-00b912bffab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:34.678974 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfef747-6906-40db-a39d-00b912bffab8 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:34.696839 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1294/2609] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:34.701212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c012f-95c9-4269-8309-530ab53b8079 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:34.712894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c012f-95c9-4269-8309-530ab53b8079 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:34.718548 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279d0fb-5ebf-448d-817f-91edc4b294c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:34.719826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d0fb-5ebf-448d-817f-91edc4b294c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:34.720192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d0fb-5ebf-448d-817f-91edc4b294c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:34.720522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d0fb-5ebf-448d-817f-91edc4b294c9 admin admin] PATH_INFO: `/v3/projects/f6def5605649401486b1f35056fe5af2/users/4cca55f5dee24c0881dbe9f5db64f05e/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:34.725471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279d0fb-5ebf-448d-817f-91edc4b294c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6def5605649401486b1f35056fe5af2, user_id=4cca55f5dee24c0881dbe9f5db64f05e, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:34.725784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279d0fb-5ebf-448d-817f-91edc4b294c9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:34.742921 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1316/2610] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:34 2022] PUT /identity/v3/projects/f6def5605649401486b1f35056fe5af2/users/4cca55f5dee24c0881dbe9f5db64f05e/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:34.911800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:35.038941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfef747-6906-40db-a39d-00b912bffab8 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:35.040825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9cfef747-6906-40db-a39d-00b912bffab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:35.162886 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:35.187966 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfef747-6906-40db-a39d-00b912bffab8 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:35.279188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f4767-f10b-4c9e-b306-547d999f022a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:35.279990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f4767-f10b-4c9e-b306-547d999f022a service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:35.280823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f4767-f10b-4c9e-b306-547d999f022a service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:35.280823 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f4767-f10b-4c9e-b306-547d999f022a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:35.283517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-857f4767-f10b-4c9e-b306-547d999f022a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:35.383858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f4767-f10b-4c9e-b306-547d999f022a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:35.386791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f4767-f10b-4c9e-b306-547d999f022a service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:35.409261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1295/2611] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:36.208739 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:36.554036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b5299-efea-4482-ace0-d97975e3867b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:36.554576 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b5299-efea-4482-ace0-d97975e3867b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:36.555074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b5299-efea-4482-ace0-d97975e3867b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:36.587333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-814b5299-efea-4482-ace0-d97975e3867b None None] MFA Rules not processed for user `4cca55f5dee24c0881dbe9f5db64f05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:36.590744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-814b5299-efea-4482-ace0-d97975e3867b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:36.708788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1317/2612] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:36 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:36.752533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:38.273367 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:38.620420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:38.648205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7264b-b665-4cdf-8344-ab7d6fad9204 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:38.648494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7264b-b665-4cdf-8344-ab7d6fad9204 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:38.648793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7264b-b665-4cdf-8344-ab7d6fad9204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:38.678160 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-08b7264b-b665-4cdf-8344-ab7d6fad9204 None None] MFA Rules not processed for user `d2f8dbd621a74ea29ff3621a49a5de32`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:38.679492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-08b7264b-b665-4cdf-8344-ab7d6fad9204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:38.785398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1296/2613] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:38.791141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:38.802209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b5299-efea-4482-ace0-d97975e3867b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:38.804180 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-814b5299-efea-4482-ace0-d97975e3867b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:38.902949 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b5299-efea-4482-ace0-d97975e3867b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:38.911965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaa480-8dba-4667-a099-09e0b4b97469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:38.912701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaa480-8dba-4667-a099-09e0b4b97469 service nova] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:38.913078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaa480-8dba-4667-a099-09e0b4b97469 service nova] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:38.913376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaa480-8dba-4667-a099-09e0b4b97469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:38.916321 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fedaa480-8dba-4667-a099-09e0b4b97469 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:38.991752 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaa480-8dba-4667-a099-09e0b4b97469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:38.993015 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaa480-8dba-4667-a099-09e0b4b97469 service nova] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:39.003292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1318/2614] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:39.057649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7264b-b665-4cdf-8344-ab7d6fad9204 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:39.131957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7264b-b665-4cdf-8344-ab7d6fad9204 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:39.136704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:39.137641 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:39.137924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:39.138197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] PATH_INFO: `/v3/users/f42c331b02dc4eb4b0c00cd6cd276496` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:39.139532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f42c331b02dc4eb4b0c00cd6cd276496)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:39.141757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:39.445031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f42c331b02dc4eb4b0c00cd6cd276496'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:39.445748 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f42c331b02dc4eb4b0c00cd6cd276496'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:39.459270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f42c331b02dc4eb4b0c00cd6cd276496'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:39.471981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f42c331b02dc4eb4b0c00cd6cd276496'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:39.528863 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:39.602263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f42c331b02dc4eb4b0c00cd6cd276496'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:39.609707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1297/2615] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:39 2022] DELETE /identity/v3/users/f42c331b02dc4eb4b0c00cd6cd276496 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:39.854258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaa480-8dba-4667-a099-09e0b4b97469 service nova] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:39.866701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaa480-8dba-4667-a099-09e0b4b97469 service nova] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:39.934460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:39.935409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:39.935679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:39.935952 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] PATH_INFO: `/v3/projects/ffdbf7f4c8014ce2b32d25ac7a88f5fb` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:39.937283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffdbf7f4c8014ce2b32d25ac7a88f5fb)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:39.939231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:40.048308 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffdbf7f4c8014ce2b32d25ac7a88f5fb', 'request_id': 'req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:40.117036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffdbf7f4c8014ce2b32d25ac7a88f5fb', 'request_id': 'req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:40.123956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffdbf7f4c8014ce2b32d25ac7a88f5fb', 'request_id': 'req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:40.143640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] The token cache is being invalidate because project ffdbf7f4c8014ce2b32d25ac7a88f5fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:40.143968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:40.144965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:40.146758 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1319/2616] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:39 2022] DELETE /identity/v3/projects/ffdbf7f4c8014ce2b32d25ac7a88f5fb => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:40.154358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:40.156018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:40.170315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e29de-9814-4162-8b42-c7808d17de15 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:40.228176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:40.229075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:40.229408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:40.229699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] PATH_INFO: `/v3/users/4cca55f5dee24c0881dbe9f5db64f05e` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:40.230899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cca55f5dee24c0881dbe9f5db64f05e)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:40.232853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:40.339907 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:40.445906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cca55f5dee24c0881dbe9f5db64f05e'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:40.446568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cca55f5dee24c0881dbe9f5db64f05e'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:40.460044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cca55f5dee24c0881dbe9f5db64f05e'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:40.468806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cca55f5dee24c0881dbe9f5db64f05e'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:40.538112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-bd6474e9-70e5-4dfb-afb1-9100c8cedf4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cca55f5dee24c0881dbe9f5db64f05e'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:40.545512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1298/2617] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:40 2022] DELETE /identity/v3/users/4cca55f5dee24c0881dbe9f5db64f05e => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:40.829140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:40.829140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:40.843070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df7b1f-67a3-478b-bd8f-d0a8cd2d0c89 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:40.909466 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:40.910247 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:40.910534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:40.910813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] PATH_INFO: `/v3/projects/f6def5605649401486b1f35056fe5af2` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:40.912032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6def5605649401486b1f35056fe5af2)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:40.914116 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:40.945320 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:41.035818 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6def5605649401486b1f35056fe5af2', 'request_id': 'req-7d852296-fb30-4979-9589-67e5c3784683'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:41.145330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6def5605649401486b1f35056fe5af2', 'request_id': 'req-7d852296-fb30-4979-9589-67e5c3784683'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:41.152436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6def5605649401486b1f35056fe5af2', 'request_id': 'req-7d852296-fb30-4979-9589-67e5c3784683'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:41.166085 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] The token cache is being invalidate because project f6def5605649401486b1f35056fe5af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:41.166648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:41.171167 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-7d852296-fb30-4979-9589-67e5c3784683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:41.174585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1320/2618] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:40 2022] DELETE /identity/v3/projects/f6def5605649401486b1f35056fe5af2 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:41.180129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f84a9-f627-4ee9-be54-4b70980feb7d None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:41.180638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f84a9-f627-4ee9-be54-4b70980feb7d None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:41.180755 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f84a9-f627-4ee9-be54-4b70980feb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:41.215212 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:41.220086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-524f84a9-f627-4ee9-be54-4b70980feb7d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:41.221577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-524f84a9-f627-4ee9-be54-4b70980feb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:41.292798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1299/2619] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:41.301922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac63c8-1e47-4580-a0c5-3d701be6bc05 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:41.302214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac63c8-1e47-4580-a0c5-3d701be6bc05 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:41.302498 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac63c8-1e47-4580-a0c5-3d701be6bc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:41.321579 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-38ac63c8-1e47-4580-a0c5-3d701be6bc05 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:41.322826 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-38ac63c8-1e47-4580-a0c5-3d701be6bc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:41.331503 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1321/2620] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:41.336136 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f84a9-f627-4ee9-be54-4b70980feb7d None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:41.346640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f84a9-f627-4ee9-be54-4b70980feb7d None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:41.351909 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62500021-0d2b-402e-962d-7cc50fc939c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:41.357368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62500021-0d2b-402e-962d-7cc50fc939c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:41.357368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62500021-0d2b-402e-962d-7cc50fc939c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:41.357368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62500021-0d2b-402e-962d-7cc50fc939c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:41.357368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62500021-0d2b-402e-962d-7cc50fc939c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:41.357368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62500021-0d2b-402e-962d-7cc50fc939c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:41.357368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62500021-0d2b-402e-962d-7cc50fc939c7 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:41.371454 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1300/2621] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:51:41.375803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac63c8-1e47-4580-a0c5-3d701be6bc05 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:41.386932 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac63c8-1e47-4580-a0c5-3d701be6bc05 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:41.397084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67df7c4-aca7-4c93-b96d-6ec13f133594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:41.397796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67df7c4-aca7-4c93-b96d-6ec13f133594 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:41.398074 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67df7c4-aca7-4c93-b96d-6ec13f133594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:41.398346 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67df7c4-aca7-4c93-b96d-6ec13f133594 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:41.399046 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67df7c4-aca7-4c93-b96d-6ec13f133594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:41.400975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67df7c4-aca7-4c93-b96d-6ec13f133594 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:41.561599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1322/2622] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:41 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:41.562926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62500021-0d2b-402e-962d-7cc50fc939c7 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:41.575427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62500021-0d2b-402e-962d-7cc50fc939c7 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:41.650865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c70e15-0653-4584-b7c1-9440e5098c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:41.651913 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c70e15-0653-4584-b7c1-9440e5098c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:41.652205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c70e15-0653-4584-b7c1-9440e5098c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:41.652486 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c70e15-0653-4584-b7c1-9440e5098c35 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:41.653249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c70e15-0653-4584-b7c1-9440e5098c35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:41.655934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c70e15-0653-4584-b7c1-9440e5098c35 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:41.750556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1301/2623] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:41 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:41.751348 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67df7c4-aca7-4c93-b96d-6ec13f133594 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:41.771379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67df7c4-aca7-4c93-b96d-6ec13f133594 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:41.778010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cb4ca-5dc0-4f05-aeb8-7001f90cf51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:41.779888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cb4ca-5dc0-4f05-aeb8-7001f90cf51a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:41.780298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cb4ca-5dc0-4f05-aeb8-7001f90cf51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:41.780701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cb4ca-5dc0-4f05-aeb8-7001f90cf51a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:41.781640 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cb4ca-5dc0-4f05-aeb8-7001f90cf51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:41.784121 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cb4ca-5dc0-4f05-aeb8-7001f90cf51a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:41.796963 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1323/2624] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:41.802882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c70e15-0653-4584-b7c1-9440e5098c35 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:41.816757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c70e15-0653-4584-b7c1-9440e5098c35 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:41.828213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d44cf-9b4c-4b7e-bf62-7d1e4cf6fc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:41.828213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d44cf-9b4c-4b7e-bf62-7d1e4cf6fc16 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:41.828213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d44cf-9b4c-4b7e-bf62-7d1e4cf6fc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:41.828213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d44cf-9b4c-4b7e-bf62-7d1e4cf6fc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:41.828732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d44cf-9b4c-4b7e-bf62-7d1e4cf6fc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:41.831313 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d44cf-9b4c-4b7e-bf62-7d1e4cf6fc16 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:41.850651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1302/2625] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:41.852351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cb4ca-5dc0-4f05-aeb8-7001f90cf51a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:41.864328 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cb4ca-5dc0-4f05-aeb8-7001f90cf51a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:41.870412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0305d0-f4cf-4d67-ab8a-3a8d770ae3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:41.871730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0305d0-f4cf-4d67-ab8a-3a8d770ae3e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:41.872344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0305d0-f4cf-4d67-ab8a-3a8d770ae3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:41.872845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0305d0-f4cf-4d67-ab8a-3a8d770ae3e2 admin admin] PATH_INFO: `/v3/projects/5c08a4ef4b4d4e368a5df4a11ca3b89e/users/19bef42d11c14fb2834908330a8ab4a5/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:41.899341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0305d0-f4cf-4d67-ab8a-3a8d770ae3e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c08a4ef4b4d4e368a5df4a11ca3b89e, user_id=19bef42d11c14fb2834908330a8ab4a5, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:41.902490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0305d0-f4cf-4d67-ab8a-3a8d770ae3e2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:42.033604 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:42.039980 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1324/2626] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:41 2022] PUT /identity/v3/projects/5c08a4ef4b4d4e368a5df4a11ca3b89e/users/19bef42d11c14fb2834908330a8ab4a5/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:42.050519 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f43590-5d4b-4507-80aa-350194133df8 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:42.050901 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f43590-5d4b-4507-80aa-350194133df8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:42.051181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f43590-5d4b-4507-80aa-350194133df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:42.086031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-06f43590-5d4b-4507-80aa-350194133df8 None None] MFA Rules not processed for user `19bef42d11c14fb2834908330a8ab4a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:42.087758 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-06f43590-5d4b-4507-80aa-350194133df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:42.158817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1303/2627] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:51:42.163223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0305d0-f4cf-4d67-ab8a-3a8d770ae3e2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:42.244025 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0305d0-f4cf-4d67-ab8a-3a8d770ae3e2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:42.251195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a817e-bf0d-480c-acc1-754c9c0992ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:42.252694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a817e-bf0d-480c-acc1-754c9c0992ef admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:42.253051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a817e-bf0d-480c-acc1-754c9c0992ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:42.253534 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a817e-bf0d-480c-acc1-754c9c0992ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:42.254426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a817e-bf0d-480c-acc1-754c9c0992ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:42.263234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a817e-bf0d-480c-acc1-754c9c0992ef admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:42.394078 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1325/2628] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:42 2022] POST /identity/v3/projects => generated 371 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:42.395928 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:42.401675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f43590-5d4b-4507-80aa-350194133df8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:42.424379 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f43590-5d4b-4507-80aa-350194133df8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:42.550488 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15533f1-a59b-4e35-810b-cb2cdcafbb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:42.551184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15533f1-a59b-4e35-810b-cb2cdcafbb58 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:42.551474 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15533f1-a59b-4e35-810b-cb2cdcafbb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:42.552987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15533f1-a59b-4e35-810b-cb2cdcafbb58 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:42.552987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15533f1-a59b-4e35-810b-cb2cdcafbb58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:42.554759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15533f1-a59b-4e35-810b-cb2cdcafbb58 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:42.631130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1304/2629] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:42 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:51:42.635606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a817e-bf0d-480c-acc1-754c9c0992ef admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:42.646767 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a817e-bf0d-480c-acc1-754c9c0992ef admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:42.660944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7d242-30c4-4bd9-bea2-a6abbd22f8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:42.661979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7d242-30c4-4bd9-bea2-a6abbd22f8fa admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:42.662428 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7d242-30c4-4bd9-bea2-a6abbd22f8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:42.662886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7d242-30c4-4bd9-bea2-a6abbd22f8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:42.663948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7d242-30c4-4bd9-bea2-a6abbd22f8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:42.666443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7d242-30c4-4bd9-bea2-a6abbd22f8fa admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:42.681301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1326/2630] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:42.683380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15533f1-a59b-4e35-810b-cb2cdcafbb58 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:42.693975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15533f1-a59b-4e35-810b-cb2cdcafbb58 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:42.698695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59298d7f-2c44-47d2-80c2-cd4e9f38232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:42.699589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59298d7f-2c44-47d2-80c2-cd4e9f38232f admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:42.699868 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59298d7f-2c44-47d2-80c2-cd4e9f38232f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:42.700140 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59298d7f-2c44-47d2-80c2-cd4e9f38232f admin admin] PATH_INFO: `/v3/projects/809fa42e8a964fc88f644c590ac6e855/users/6429f92b28894cc695a65a16eead4c3c/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:42.717357 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59298d7f-2c44-47d2-80c2-cd4e9f38232f admin admin] RBAC: Authorizing `identity:create_grant(project_id=809fa42e8a964fc88f644c590ac6e855, user_id=6429f92b28894cc695a65a16eead4c3c, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:42.720553 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59298d7f-2c44-47d2-80c2-cd4e9f38232f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:42.831636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1305/2631] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:42 2022] PUT /identity/v3/projects/809fa42e8a964fc88f644c590ac6e855/users/6429f92b28894cc695a65a16eead4c3c/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:42.835714 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7d242-30c4-4bd9-bea2-a6abbd22f8fa admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:42.846636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7d242-30c4-4bd9-bea2-a6abbd22f8fa admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:42.916211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8cf7f-abfb-46b4-bb15-8557031f3f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:42.916978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8cf7f-abfb-46b4-bb15-8557031f3f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:42.917417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8cf7f-abfb-46b4-bb15-8557031f3f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:42.917788 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8cf7f-abfb-46b4-bb15-8557031f3f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:42.918614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8cf7f-abfb-46b4-bb15-8557031f3f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:42.920291 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8cf7f-abfb-46b4-bb15-8557031f3f52 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:42.931572 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1327/2632] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:42.935890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59298d7f-2c44-47d2-80c2-cd4e9f38232f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:42.946862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59298d7f-2c44-47d2-80c2-cd4e9f38232f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:42.952037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf6c96-1af6-449b-a41d-f34ed2b8d7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:42.952946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf6c96-1af6-449b-a41d-f34ed2b8d7d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:42.953414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf6c96-1af6-449b-a41d-f34ed2b8d7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:42.953776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf6c96-1af6-449b-a41d-f34ed2b8d7d5 admin admin] PATH_INFO: `/v3/projects/809fa42e8a964fc88f644c590ac6e855/users/6429f92b28894cc695a65a16eead4c3c/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:42.956348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf6c96-1af6-449b-a41d-f34ed2b8d7d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=809fa42e8a964fc88f644c590ac6e855, user_id=6429f92b28894cc695a65a16eead4c3c, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:42.959224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf6c96-1af6-449b-a41d-f34ed2b8d7d5 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:42.984721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a2ea6-ee89-4037-9ec8-123f96c47a93 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:42.985239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a2ea6-ee89-4037-9ec8-123f96c47a93 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:42.985758 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a2ea6-ee89-4037-9ec8-123f96c47a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:42.988121 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1306/2633] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:42 2022] PUT /identity/v3/projects/809fa42e8a964fc88f644c590ac6e855/users/6429f92b28894cc695a65a16eead4c3c/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:43.017897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fa1a2ea6-ee89-4037-9ec8-123f96c47a93 None None] MFA Rules not processed for user `6429f92b28894cc695a65a16eead4c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:43.019169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fa1a2ea6-ee89-4037-9ec8-123f96c47a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:43.146977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:43.150851 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1328/2634] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:43.353272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf6c96-1af6-449b-a41d-f34ed2b8d7d5 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:43.354996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ffbf6c96-1af6-449b-a41d-f34ed2b8d7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:43.412749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf6c96-1af6-449b-a41d-f34ed2b8d7d5 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:43.416335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59101a3-c054-4f00-aa33-ce9f4fb19a42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:43.416978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59101a3-c054-4f00-aa33-ce9f4fb19a42 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:43.417290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59101a3-c054-4f00-aa33-ce9f4fb19a42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:43.417563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59101a3-c054-4f00-aa33-ce9f4fb19a42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:43.427130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59101a3-c054-4f00-aa33-ce9f4fb19a42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:43.428598 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59101a3-c054-4f00-aa33-ce9f4fb19a42 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:43.438399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1307/2635] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:43.768238 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a2ea6-ee89-4037-9ec8-123f96c47a93 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:43.778276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a2ea6-ee89-4037-9ec8-123f96c47a93 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:43.782026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f71569-90ec-4c71-b93a-56e8e02e3a20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:43.782732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f71569-90ec-4c71-b93a-56e8e02e3a20 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:43.783019 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f71569-90ec-4c71-b93a-56e8e02e3a20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:43.783455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f71569-90ec-4c71-b93a-56e8e02e3a20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:43.849463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f71569-90ec-4c71-b93a-56e8e02e3a20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:43.850908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f71569-90ec-4c71-b93a-56e8e02e3a20 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:43.861744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1329/2636] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:44.238377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:44.538679 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:45.406273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:45.497820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:45.823957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:45.934097 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:45.991233 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:46.185142 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:46.322494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:48.000975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59101a3-c054-4f00-aa33-ce9f4fb19a42 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:48.000975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b59101a3-c054-4f00-aa33-ce9f4fb19a42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:48.072163 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59101a3-c054-4f00-aa33-ce9f4fb19a42 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:48.080040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:48.081542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:48.090989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:48.090989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] PATH_INFO: `/v3/users/72f9a39f2899442cb20514bff3c46dda` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:48.090989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f9a39f2899442cb20514bff3c46dda)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:48.090989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:48.329992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f9a39f2899442cb20514bff3c46dda'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:48.330647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f9a39f2899442cb20514bff3c46dda'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:48.343548 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f9a39f2899442cb20514bff3c46dda'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:48.350997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f9a39f2899442cb20514bff3c46dda'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:48.463842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f9a39f2899442cb20514bff3c46dda'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:48.470347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1308/2637] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:47 2022] DELETE /identity/v3/users/72f9a39f2899442cb20514bff3c46dda => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:48.475869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b821-2643-470f-afb0-824bb5ed458a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:48.476366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b821-2643-470f-afb0-824bb5ed458a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:48.476554 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115b821-2643-470f-afb0-824bb5ed458a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:48.495577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-d115b821-2643-470f-afb0-824bb5ed458a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:48.496792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d115b821-2643-470f-afb0-824bb5ed458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:48.563268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1330/2638] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:48.573666 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:48.575331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:48.641114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bd083-5f6c-4a36-9f6d-7dc3e898f18d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:48.648468 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df53b1-1783-44ed-a421-55b44eda8f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:48.649449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df53b1-1783-44ed-a421-55b44eda8f7e service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:48.649885 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df53b1-1783-44ed-a421-55b44eda8f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:48.651361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df53b1-1783-44ed-a421-55b44eda8f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:48.661829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df53b1-1783-44ed-a421-55b44eda8f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:48.663326 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df53b1-1783-44ed-a421-55b44eda8f7e service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:48.675096 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1309/2639] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:51:49.309552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115b821-2643-470f-afb0-824bb5ed458a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:49.310921 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d115b821-2643-470f-afb0-824bb5ed458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:49.326110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115b821-2643-470f-afb0-824bb5ed458a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:49.344137 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:49.345029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:49.345392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:49.345813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] PATH_INFO: `/v3/projects/47770a29807b44ba8c74c2b7d3b47a83` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:49.347923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47770a29807b44ba8c74c2b7d3b47a83)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:49.349956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:49.542273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47770a29807b44ba8c74c2b7d3b47a83', 'request_id': 'req-95ce8b3b-9411-4010-9846-4a3fd9eeb379'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:49.818212 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47770a29807b44ba8c74c2b7d3b47a83', 'request_id': 'req-95ce8b3b-9411-4010-9846-4a3fd9eeb379'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:49.828808 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47770a29807b44ba8c74c2b7d3b47a83', 'request_id': 'req-95ce8b3b-9411-4010-9846-4a3fd9eeb379'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:49.842284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] The token cache is being invalidate because project 47770a29807b44ba8c74c2b7d3b47a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:49.842616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:49.843271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:49.851106 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1331/2640] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:49 2022] DELETE /identity/v3/projects/47770a29807b44ba8c74c2b7d3b47a83 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:49.852611 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df53b1-1783-44ed-a421-55b44eda8f7e service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:49.882079 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df53b1-1783-44ed-a421-55b44eda8f7e service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:49.910008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:49.954257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:49.955648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:49.956589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:49.956729 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] PATH_INFO: `/v3/users/eb5a667fa9c24fda8cdb8e99e74066bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:49.958707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb5a667fa9c24fda8cdb8e99e74066bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:49.962060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:50.193126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb5a667fa9c24fda8cdb8e99e74066bd'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.194051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb5a667fa9c24fda8cdb8e99e74066bd'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.213419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb5a667fa9c24fda8cdb8e99e74066bd'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.220858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb5a667fa9c24fda8cdb8e99e74066bd'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.322563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb5a667fa9c24fda8cdb8e99e74066bd'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.337091 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1310/2641] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:49 2022] DELETE /identity/v3/users/eb5a667fa9c24fda8cdb8e99e74066bd => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:50.338417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:50.340067 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:50.354627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce8b3b-9411-4010-9846-4a3fd9eeb379 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:50.359297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:50.418311 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:50.419188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:50.419468 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:50.419737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] PATH_INFO: `/v3/users/d1d028c346ff4b67b4a3067a41cbd68f` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:50.421043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d028c346ff4b67b4a3067a41cbd68f)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:50.422871 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:50.576389 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d028c346ff4b67b4a3067a41cbd68f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.585177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d028c346ff4b67b4a3067a41cbd68f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.605433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d028c346ff4b67b4a3067a41cbd68f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.687986 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d028c346ff4b67b4a3067a41cbd68f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.688605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d028c346ff4b67b4a3067a41cbd68f'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.695330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1332/2642] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:50 2022] DELETE /identity/v3/users/d1d028c346ff4b67b4a3067a41cbd68f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:50.779141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:50.780847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:50.798075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29126df1-e3e0-4511-9597-ed9b0448b87b admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:50.867156 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:50.868254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:50.868536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:50.868816 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] PATH_INFO: `/v3/projects/eddd674a53754221ba3a333c0c065d1b` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:50.870258 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eddd674a53754221ba3a333c0c065d1b)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:50.872225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:50.972829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:50.975123 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:50.980718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:50.982731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eddd674a53754221ba3a333c0c065d1b', 'request_id': 'req-66ffedfb-c5bf-433b-8a00-bc9c601025e9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:50.991073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eddd674a53754221ba3a333c0c065d1b', 'request_id': 'req-66ffedfb-c5bf-433b-8a00-bc9c601025e9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.000896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eddd674a53754221ba3a333c0c065d1b', 'request_id': 'req-66ffedfb-c5bf-433b-8a00-bc9c601025e9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.014677 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0070d6-d04d-467f-989b-90bff2c5448f admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:51.074970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] The token cache is being invalidate because project eddd674a53754221ba3a333c0c065d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:51.075351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.076105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-66ffedfb-c5bf-433b-8a00-bc9c601025e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.077632 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:51.077791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1311/2643] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:50 2022] DELETE /identity/v3/projects/eddd674a53754221ba3a333c0c065d1b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:51.078542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.078821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.079090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] PATH_INFO: `/v3/projects/9319e063b19341d5b5bdcc121363a476` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.080289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9319e063b19341d5b5bdcc121363a476)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:51.082391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:51.083791 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9baf80-7abf-4afb-932f-aad60e6284eb None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.084119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9baf80-7abf-4afb-932f-aad60e6284eb None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.084396 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9baf80-7abf-4afb-932f-aad60e6284eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.103963 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-5e9baf80-7abf-4afb-932f-aad60e6284eb None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:51.105260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5e9baf80-7abf-4afb-932f-aad60e6284eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:51.114177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1312/2644] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:51.122151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96925a-b9f0-4228-88e6-f772eb79a575 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.122443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96925a-b9f0-4228-88e6-f772eb79a575 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.122717 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96925a-b9f0-4228-88e6-f772eb79a575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.141061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-9e96925a-b9f0-4228-88e6-f772eb79a575 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:51.142353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-9e96925a-b9f0-4228-88e6-f772eb79a575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:51.151189 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1313/2645] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:51:51.155549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96925a-b9f0-4228-88e6-f772eb79a575 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:51.165268 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96925a-b9f0-4228-88e6-f772eb79a575 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:51.169856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776cdb9-c47b-41aa-990a-39edba1825a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:51.170493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776cdb9-c47b-41aa-990a-39edba1825a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.170776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776cdb9-c47b-41aa-990a-39edba1825a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.171051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776cdb9-c47b-41aa-990a-39edba1825a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.171821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776cdb9-c47b-41aa-990a-39edba1825a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:51.172132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776cdb9-c47b-41aa-990a-39edba1825a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:51.173757 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776cdb9-c47b-41aa-990a-39edba1825a2 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:51.182254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:51.187138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1314/2646] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:51:51.192202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776cdb9-c47b-41aa-990a-39edba1825a2 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:51.202205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776cdb9-c47b-41aa-990a-39edba1825a2 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:51.208712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9319e063b19341d5b5bdcc121363a476', 'request_id': 'req-b364eca4-c6ff-49c6-a8e4-9643cf32c817'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.264167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afc099-e792-4587-ad7c-a79afd9b1dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:51.264827 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afc099-e792-4587-ad7c-a79afd9b1dae admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.265111 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afc099-e792-4587-ad7c-a79afd9b1dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.265414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afc099-e792-4587-ad7c-a79afd9b1dae admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.266158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afc099-e792-4587-ad7c-a79afd9b1dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:51.268115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afc099-e792-4587-ad7c-a79afd9b1dae admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:51.305470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9319e063b19341d5b5bdcc121363a476', 'request_id': 'req-b364eca4-c6ff-49c6-a8e4-9643cf32c817'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.317643 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9319e063b19341d5b5bdcc121363a476', 'request_id': 'req-b364eca4-c6ff-49c6-a8e4-9643cf32c817'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.331029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] The token cache is being invalidate because project 9319e063b19341d5b5bdcc121363a476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:51.331248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.332577 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.334608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1333/2647] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:50 2022] DELETE /identity/v3/projects/9319e063b19341d5b5bdcc121363a476 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:51.340054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:51.341538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:51.360203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1315/2648] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:51 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:51.360573 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364eca4-c6ff-49c6-a8e4-9643cf32c817 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:51.360987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afc099-e792-4587-ad7c-a79afd9b1dae admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:51.363309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-37afc099-e792-4587-ad7c-a79afd9b1dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:51.432478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:51.433597 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.433879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.434150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] PATH_INFO: `/v3/users/4a87961bce7e4333bbe1dc29dfce9617` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.435648 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a87961bce7e4333bbe1dc29dfce9617)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:51.438161 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:51.451815 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afc099-e792-4587-ad7c-a79afd9b1dae admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:51.456870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383c74d-65c9-464e-88ee-c971d0c63713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:51.461417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383c74d-65c9-464e-88ee-c971d0c63713 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.461417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383c74d-65c9-464e-88ee-c971d0c63713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.461417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383c74d-65c9-464e-88ee-c971d0c63713 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.461417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383c74d-65c9-464e-88ee-c971d0c63713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:51.461417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383c74d-65c9-464e-88ee-c971d0c63713 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:51.577360 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1316/2649] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:51 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:51.579306 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383c74d-65c9-464e-88ee-c971d0c63713 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:51.589464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383c74d-65c9-464e-88ee-c971d0c63713 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:51.594567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa359-c665-4b36-a461-dd1f396daae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:51.595256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa359-c665-4b36-a461-dd1f396daae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.595535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa359-c665-4b36-a461-dd1f396daae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.595911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa359-c665-4b36-a461-dd1f396daae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.597031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aa359-c665-4b36-a461-dd1f396daae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:51.598830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aa359-c665-4b36-a461-dd1f396daae9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:51.610144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1317/2650] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:51.623492 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa359-c665-4b36-a461-dd1f396daae9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:51.633761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa359-c665-4b36-a461-dd1f396daae9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:51.638891 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4d720-dcd7-4591-8b13-77ba7107774f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:51.639565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4d720-dcd7-4591-8b13-77ba7107774f admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.639848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4d720-dcd7-4591-8b13-77ba7107774f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.640185 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4d720-dcd7-4591-8b13-77ba7107774f admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.640933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4d720-dcd7-4591-8b13-77ba7107774f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:51.643001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4d720-dcd7-4591-8b13-77ba7107774f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:51.644491 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:51.648578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a87961bce7e4333bbe1dc29dfce9617'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.657495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1318/2651] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:51:51.661477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a87961bce7e4333bbe1dc29dfce9617'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.662369 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4d720-dcd7-4591-8b13-77ba7107774f admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:51.672441 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4d720-dcd7-4591-8b13-77ba7107774f admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:51.673366 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a87961bce7e4333bbe1dc29dfce9617'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.678181 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0d7e3-0419-480a-85ce-fc126e3ba6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:51.679249 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0d7e3-0419-480a-85ce-fc126e3ba6e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.679384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0d7e3-0419-480a-85ce-fc126e3ba6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.679588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0d7e3-0419-480a-85ce-fc126e3ba6e3 admin admin] PATH_INFO: `/v3/projects/3de56edd4b0140b692b4e4f62deb5bd4/users/0b2f6947d03243b58caf3bc9243ca9bc/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.694590 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0d7e3-0419-480a-85ce-fc126e3ba6e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de56edd4b0140b692b4e4f62deb5bd4, user_id=0b2f6947d03243b58caf3bc9243ca9bc, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:51.697399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0d7e3-0419-480a-85ce-fc126e3ba6e3 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:51.745284 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a87961bce7e4333bbe1dc29dfce9617'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.745913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dcc66a0b-67f5-4684-b11c-45a5ac5123d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a87961bce7e4333bbe1dc29dfce9617'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:51.752340 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1334/2652] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:51:51 2022] DELETE /identity/v3/users/4a87961bce7e4333bbe1dc29dfce9617 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:51.774329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1319/2653] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:51 2022] PUT /identity/v3/projects/3de56edd4b0140b692b4e4f62deb5bd4/users/0b2f6947d03243b58caf3bc9243ca9bc/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:51:51.787335 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c80046-817e-4324-8ea2-ef8b6548e79d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:51.787626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c80046-817e-4324-8ea2-ef8b6548e79d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:51.787903 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c80046-817e-4324-8ea2-ef8b6548e79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:51.943273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-81c80046-817e-4324-8ea2-ef8b6548e79d None None] MFA Rules not processed for user `0b2f6947d03243b58caf3bc9243ca9bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:51.944521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-81c80046-817e-4324-8ea2-ef8b6548e79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:51.968482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:52.024277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1335/2654] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:52.032332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0d7e3-0419-480a-85ce-fc126e3ba6e3 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:52.035138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4ba0d7e3-0419-480a-85ce-fc126e3ba6e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:52.037601 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c80046-817e-4324-8ea2-ef8b6548e79d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:52.039090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-81c80046-817e-4324-8ea2-ef8b6548e79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:52.104945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c80046-817e-4324-8ea2-ef8b6548e79d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:52.108574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e8e9e-80c4-4dda-a599-59e13c3ffb39 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1a7213cbf6448b7ba6570a52a4c7c0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:52.109427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e8e9e-80c4-4dda-a599-59e13c3ffb39 service glance] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:52.109689 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e8e9e-80c4-4dda-a599-59e13c3ffb39 service glance] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:52.109968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e8e9e-80c4-4dda-a599-59e13c3ffb39 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:52.136423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e8e9e-80c4-4dda-a599-59e13c3ffb39 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:52.138874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e8e9e-80c4-4dda-a599-59e13c3ffb39 service glance] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:52.148011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1336/2655] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:51:52.189431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0d7e3-0419-480a-85ce-fc126e3ba6e3 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:52.194383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:52.195756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:52.195756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:52.195756 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] PATH_INFO: `/v3/projects/ba91797bcd2a4c62855828e00c7053ed` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:52.196983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba91797bcd2a4c62855828e00c7053ed)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:52.198937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:52.352834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba91797bcd2a4c62855828e00c7053ed', 'request_id': 'req-44a48fba-eb1b-4dcd-8740-5f85e278ad74'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:52.359109 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba91797bcd2a4c62855828e00c7053ed', 'request_id': 'req-44a48fba-eb1b-4dcd-8740-5f85e278ad74'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:52.367082 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba91797bcd2a4c62855828e00c7053ed', 'request_id': 'req-44a48fba-eb1b-4dcd-8740-5f85e278ad74'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:52.470779 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] The token cache is being invalidate because project ba91797bcd2a4c62855828e00c7053ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:51:52.471075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:52.471778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-44a48fba-eb1b-4dcd-8740-5f85e278ad74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:51:52.473609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1320/2656] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:51:52 2022] DELETE /identity/v3/projects/ba91797bcd2a4c62855828e00c7053ed => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:52.479356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3cb0a-b205-4f14-96d0-857a3cb0312b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:52.479646 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3cb0a-b205-4f14-96d0-857a3cb0312b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:52.479996 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3cb0a-b205-4f14-96d0-857a3cb0312b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:52.539533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-cbd3cb0a-b205-4f14-96d0-857a3cb0312b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:52.540800 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-cbd3cb0a-b205-4f14-96d0-857a3cb0312b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:52.601843 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1337/2657] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:52.610004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f1e48-5ae3-4e05-b0cc-1fec48efdc86 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:52.610302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f1e48-5ae3-4e05-b0cc-1fec48efdc86 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:52.610605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f1e48-5ae3-4e05-b0cc-1fec48efdc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:52.632685 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-922f1e48-5ae3-4e05-b0cc-1fec48efdc86 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:52.634037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-922f1e48-5ae3-4e05-b0cc-1fec48efdc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:52.643176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1321/2658] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:52.649376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3cb0a-b205-4f14-96d0-857a3cb0312b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:52.665449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3cb0a-b205-4f14-96d0-857a3cb0312b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:52.672062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9339308-4aae-4183-be6d-397f0ee8e12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:52.672720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9339308-4aae-4183-be6d-397f0ee8e12b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:52.673142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9339308-4aae-4183-be6d-397f0ee8e12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:52.673345 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9339308-4aae-4183-be6d-397f0ee8e12b admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:52.674075 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9339308-4aae-4183-be6d-397f0ee8e12b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:51:52.674380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9339308-4aae-4183-be6d-397f0ee8e12b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:52.676493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9339308-4aae-4183-be6d-397f0ee8e12b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:52.683611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=100 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:52.688991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1338/2659] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:51:52.694139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f1e48-5ae3-4e05-b0cc-1fec48efdc86 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:52.698761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:51:52.704274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f1e48-5ae3-4e05-b0cc-1fec48efdc86 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:52.709771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d027ca-5dde-40b2-bd5d-6a80c05b5368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:52.710663 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d027ca-5dde-40b2-bd5d-6a80c05b5368 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:52.710906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d027ca-5dde-40b2-bd5d-6a80c05b5368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:52.711221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d027ca-5dde-40b2-bd5d-6a80c05b5368 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:52.712024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d027ca-5dde-40b2-bd5d-6a80c05b5368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:52.714150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d027ca-5dde-40b2-bd5d-6a80c05b5368 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:52.808826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1322/2660] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:51:52 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:52.813357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9339308-4aae-4183-be6d-397f0ee8e12b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:52.823913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9339308-4aae-4183-be6d-397f0ee8e12b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:52.887852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0227c-06bd-4134-967a-e8ced7fd339d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:52.888478 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0227c-06bd-4134-967a-e8ced7fd339d admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:52.888760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0227c-06bd-4134-967a-e8ced7fd339d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:52.889034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0227c-06bd-4134-967a-e8ced7fd339d admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:52.889930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0227c-06bd-4134-967a-e8ced7fd339d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:52.891894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0227c-06bd-4134-967a-e8ced7fd339d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:53.014539 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1339/2661] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:51:52 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:51:53.018790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d027ca-5dde-40b2-bd5d-6a80c05b5368 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:53.031064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d027ca-5dde-40b2-bd5d-6a80c05b5368 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:53.036750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e7778-2473-432e-bae9-5f4886787aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:53.037457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e7778-2473-432e-bae9-5f4886787aaf admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:53.037743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e7778-2473-432e-bae9-5f4886787aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:53.038023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e7778-2473-432e-bae9-5f4886787aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:53.038896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e7778-2473-432e-bae9-5f4886787aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:53.040617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e7778-2473-432e-bae9-5f4886787aaf admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:53.052428 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1323/2662] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:53.056395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0227c-06bd-4134-967a-e8ced7fd339d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:53.067111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0227c-06bd-4134-967a-e8ced7fd339d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:53.072385 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fea28-6a06-4cfa-be78-46fbbc52ebaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:53.073092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fea28-6a06-4cfa-be78-46fbbc52ebaa admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:53.073420 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fea28-6a06-4cfa-be78-46fbbc52ebaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:53.073695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fea28-6a06-4cfa-be78-46fbbc52ebaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:53.074568 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fea28-6a06-4cfa-be78-46fbbc52ebaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:53.076295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fea28-6a06-4cfa-be78-46fbbc52ebaa admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:53.087755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1340/2663] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:51:53.092037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e7778-2473-432e-bae9-5f4886787aaf admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:53.102680 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e7778-2473-432e-bae9-5f4886787aaf admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:53.107517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3925072-d7da-4dbd-9346-5e0f279a2c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:53.108398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3925072-d7da-4dbd-9346-5e0f279a2c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:53.108679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3925072-d7da-4dbd-9346-5e0f279a2c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:53.109069 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3925072-d7da-4dbd-9346-5e0f279a2c72 admin admin] PATH_INFO: `/v3/projects/9940fcecf9754f8e8cb0dac2188f1322/users/db778953711c45a2bcd0ec75b8105266/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:53.126000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3925072-d7da-4dbd-9346-5e0f279a2c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9940fcecf9754f8e8cb0dac2188f1322, user_id=db778953711c45a2bcd0ec75b8105266, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:53.129482 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3925072-d7da-4dbd-9346-5e0f279a2c72 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:53.207286 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1324/2664] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:51:53 2022] PUT /identity/v3/projects/9940fcecf9754f8e8cb0dac2188f1322/users/db778953711c45a2bcd0ec75b8105266/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:51:53.218435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b7164-fb57-47c9-95ed-15f2270e689f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:53.218725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b7164-fb57-47c9-95ed-15f2270e689f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:53.219006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b7164-fb57-47c9-95ed-15f2270e689f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:53.247991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-107b7164-fb57-47c9-95ed-15f2270e689f None None] MFA Rules not processed for user `db778953711c45a2bcd0ec75b8105266`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:51:53.249243 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-107b7164-fb57-47c9-95ed-15f2270e689f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:53.319516 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1341/2665] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:51:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:51:53.336045 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3925072-d7da-4dbd-9346-5e0f279a2c72 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:51:53.337961 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a3925072-d7da-4dbd-9346-5e0f279a2c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:51:53.423278 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3925072-d7da-4dbd-9346-5e0f279a2c72 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:51:53.427282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65a31b-25df-4a0b-bb57-e1d064e21916 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:51:53.427931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65a31b-25df-4a0b-bb57-e1d064e21916 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:51:53.428209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65a31b-25df-4a0b-bb57-e1d064e21916 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:51:53.428483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65a31b-25df-4a0b-bb57-e1d064e21916 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:51:53.437271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65a31b-25df-4a0b-bb57-e1d064e21916 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:51:53.438708 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65a31b-25df-4a0b-bb57-e1d064e21916 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:51:53.447738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1325/2666] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:51:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:01.293977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b7164-fb57-47c9-95ed-15f2270e689f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:01.295356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-107b7164-fb57-47c9-95ed-15f2270e689f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:01.368967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:01.398906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b7164-fb57-47c9-95ed-15f2270e689f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:01.405433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e76d3-d07b-48d8-8c3f-b8ecacc0a44c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:01.408127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e76d3-d07b-48d8-8c3f-b8ecacc0a44c service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:01.408127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e76d3-d07b-48d8-8c3f-b8ecacc0a44c service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:01.408127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e76d3-d07b-48d8-8c3f-b8ecacc0a44c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:01.411397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-da9e76d3-d07b-48d8-8c3f-b8ecacc0a44c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:01.426129 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e76d3-d07b-48d8-8c3f-b8ecacc0a44c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:01.428329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e76d3-d07b-48d8-8c3f-b8ecacc0a44c service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:01.440774 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1342/2667] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:03.093402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65a31b-25df-4a0b-bb57-e1d064e21916 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:03.094937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1b65a31b-25df-4a0b-bb57-e1d064e21916 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:03.174259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65a31b-25df-4a0b-bb57-e1d064e21916 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:03.179704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:03.180617 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:03.180902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:03.181200 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] PATH_INFO: `/v3/users/057fd9ecc91d40edbd05da84e22fafa8` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:03.182506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=057fd9ecc91d40edbd05da84e22fafa8)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:03.184451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:03.430629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:03.439890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '057fd9ecc91d40edbd05da84e22fafa8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:03.440605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '057fd9ecc91d40edbd05da84e22fafa8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:03.453361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '057fd9ecc91d40edbd05da84e22fafa8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:03.461809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '057fd9ecc91d40edbd05da84e22fafa8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:03.511006 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '057fd9ecc91d40edbd05da84e22fafa8'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:03.518669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1326/2668] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:03 2022] DELETE /identity/v3/users/057fd9ecc91d40edbd05da84e22fafa8 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:03.524359 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6705fbf-d39e-477f-832a-6530861d97ca None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:03.524923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6705fbf-d39e-477f-832a-6530861d97ca None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:03.524923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6705fbf-d39e-477f-832a-6530861d97ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:03.544181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b6705fbf-d39e-477f-832a-6530861d97ca None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:03.545471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b6705fbf-d39e-477f-832a-6530861d97ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:03.609416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1343/2669] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:03.616580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:03.618225 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:03.679243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd7fb0-179b-42ba-b8ff-3c716b382b7a admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:03.683209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d89769-3b97-4194-9770-d32f87ae3ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:03.683873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d89769-3b97-4194-9770-d32f87ae3ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:03.684159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d89769-3b97-4194-9770-d32f87ae3ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:03.684462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d89769-3b97-4194-9770-d32f87ae3ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:03.694764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d89769-3b97-4194-9770-d32f87ae3ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:03.698356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d89769-3b97-4194-9770-d32f87ae3ef4 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:03.706530 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1327/2670] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:04.158482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6705fbf-d39e-477f-832a-6530861d97ca None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:04.160016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b6705fbf-d39e-477f-832a-6530861d97ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:04.172865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6705fbf-d39e-477f-832a-6530861d97ca None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:04.178235 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:04.182052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:04.182052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:04.182052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] PATH_INFO: `/v3/projects/1b1b092e5cbb435ba9b9e6c0b2824cdf` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:04.182052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b1b092e5cbb435ba9b9e6c0b2824cdf)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:04.183250 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:04.334669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b1b092e5cbb435ba9b9e6c0b2824cdf', 'request_id': 'req-be1cdec9-365d-433c-b6df-56ab8f49e401'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:04.407553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b1b092e5cbb435ba9b9e6c0b2824cdf', 'request_id': 'req-be1cdec9-365d-433c-b6df-56ab8f49e401'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:04.414268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b1b092e5cbb435ba9b9e6c0b2824cdf', 'request_id': 'req-be1cdec9-365d-433c-b6df-56ab8f49e401'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:04.427683 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] The token cache is being invalidate because project 1b1b092e5cbb435ba9b9e6c0b2824cdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:04.428034 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:04.428709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:04.430265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1344/2671] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:04 2022] DELETE /identity/v3/projects/1b1b092e5cbb435ba9b9e6c0b2824cdf => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:04.437646 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d89769-3b97-4194-9770-d32f87ae3ef4 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:04.446957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d89769-3b97-4194-9770-d32f87ae3ef4 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:04.504352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:04.505247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:04.505535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:04.505824 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] PATH_INFO: `/v3/users/8d18d4ac05c245e88c3a57846065125b` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:04.507033 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d18d4ac05c245e88c3a57846065125b)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:04.509037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:04.737053 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d18d4ac05c245e88c3a57846065125b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:04.737790 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d18d4ac05c245e88c3a57846065125b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:04.745261 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:04.750678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d18d4ac05c245e88c3a57846065125b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:04.757906 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d18d4ac05c245e88c3a57846065125b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:04.842459 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-a82021f7-49c4-4994-b658-e774fa016c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d18d4ac05c245e88c3a57846065125b'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:04.849743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1328/2672] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:04 2022] DELETE /identity/v3/users/8d18d4ac05c245e88c3a57846065125b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:05.289906 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:05.291287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:05.319218 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cdec9-365d-433c-b6df-56ab8f49e401 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:05.381032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:05.381923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:05.382205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:05.382479 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] PATH_INFO: `/v3/projects/d950a314f32e441c8ad7d87b8e748507` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:05.383694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d950a314f32e441c8ad7d87b8e748507)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:05.385652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:05.387324 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:05.493489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd950a314f32e441c8ad7d87b8e748507', 'request_id': 'req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:05.572977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd950a314f32e441c8ad7d87b8e748507', 'request_id': 'req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:05.581641 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd950a314f32e441c8ad7d87b8e748507', 'request_id': 'req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:05.594064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] The token cache is being invalidate because project d950a314f32e441c8ad7d87b8e748507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:05.599244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:05.599244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-785e168b-3cbc-48c9-b3e4-a78c5cf507f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:05.599244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1345/2673] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:05 2022] DELETE /identity/v3/projects/d950a314f32e441c8ad7d87b8e748507 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:05.605193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a2f7e-810f-48cf-8927-a1db5997aab8 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:05.605193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a2f7e-810f-48cf-8927-a1db5997aab8 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:05.605193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a2f7e-810f-48cf-8927-a1db5997aab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:05.628247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-da5a2f7e-810f-48cf-8927-a1db5997aab8 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:05.628247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-da5a2f7e-810f-48cf-8927-a1db5997aab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:05.696415 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1329/2674] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:05.704217 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12569072-715c-4670-91e5-bdc9a360a4ff None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:05.708504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12569072-715c-4670-91e5-bdc9a360a4ff None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:05.708504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12569072-715c-4670-91e5-bdc9a360a4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:05.732084 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a2f7e-810f-48cf-8927-a1db5997aab8 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:05.733522 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-da5a2f7e-810f-48cf-8927-a1db5997aab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:05.737252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-12569072-715c-4670-91e5-bdc9a360a4ff None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:05.738433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-12569072-715c-4670-91e5-bdc9a360a4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:05.747140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1346/2675] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:05.748588 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a2f7e-810f-48cf-8927-a1db5997aab8 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:05.751651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12569072-715c-4670-91e5-bdc9a360a4ff None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:05.753990 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:05.755408 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:05.755690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:05.755963 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] PATH_INFO: `/v3/users/db778953711c45a2bcd0ec75b8105266` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:05.757219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] RBAC: Authorizing `identity:delete_user(user_id=db778953711c45a2bcd0ec75b8105266)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:05.761638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:05.761902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12569072-715c-4670-91e5-bdc9a360a4ff None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:05.766793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fb97c-85a2-4926-b3ae-a39bbf814390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:05.767459 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fb97c-85a2-4926-b3ae-a39bbf814390 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:05.767763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fb97c-85a2-4926-b3ae-a39bbf814390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:05.768062 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fb97c-85a2-4926-b3ae-a39bbf814390 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:05.768754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fb97c-85a2-4926-b3ae-a39bbf814390 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:52:05.769066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fb97c-85a2-4926-b3ae-a39bbf814390 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:05.771301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fb97c-85a2-4926-b3ae-a39bbf814390 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:05.789412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1347/2676] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:52:05.790604 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fb97c-85a2-4926-b3ae-a39bbf814390 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:05.804925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fb97c-85a2-4926-b3ae-a39bbf814390 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:05.809853 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d6e62-8ce8-4cea-8121-97d2365371bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:05.810584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d6e62-8ce8-4cea-8121-97d2365371bf admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:05.810805 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d6e62-8ce8-4cea-8121-97d2365371bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:05.811080 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d6e62-8ce8-4cea-8121-97d2365371bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:05.811772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d6e62-8ce8-4cea-8121-97d2365371bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:05.813747 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d6e62-8ce8-4cea-8121-97d2365371bf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:05.935113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1348/2677] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:05 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:05.939304 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d6e62-8ce8-4cea-8121-97d2365371bf admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:05.970942 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db778953711c45a2bcd0ec75b8105266'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:05.971752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db778953711c45a2bcd0ec75b8105266'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:05.984643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db778953711c45a2bcd0ec75b8105266'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:05.992004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db778953711c45a2bcd0ec75b8105266'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:06.001630 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d6e62-8ce8-4cea-8121-97d2365371bf admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:06.006878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440c527-c232-40cd-98f4-cf59d6160b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:06.007545 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c527-c232-40cd-98f4-cf59d6160b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.007828 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c527-c232-40cd-98f4-cf59d6160b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.008095 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c527-c232-40cd-98f4-cf59d6160b5e admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.008781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440c527-c232-40cd-98f4-cf59d6160b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:06.010703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440c527-c232-40cd-98f4-cf59d6160b5e admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:06.059019 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-17834484-c536-4e70-8124-fe4a2afdc76b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db778953711c45a2bcd0ec75b8105266'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:06.070477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1330/2678] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:05 2022] DELETE /identity/v3/users/db778953711c45a2bcd0ec75b8105266 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:06.075078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f732e-d0ff-402a-9ab5-9e4747552087 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.077289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f732e-d0ff-402a-9ab5-9e4747552087 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.077606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f732e-d0ff-402a-9ab5-9e4747552087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.090610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1349/2679] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:05 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:52:06.099551 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-517f732e-d0ff-402a-9ab5-9e4747552087 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:06.100775 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-517f732e-d0ff-402a-9ab5-9e4747552087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:06.101647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440c527-c232-40cd-98f4-cf59d6160b5e admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:06.104055 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b440c527-c232-40cd-98f4-cf59d6160b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:06.131379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440c527-c232-40cd-98f4-cf59d6160b5e admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:06.172584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:06.180331 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:06.196770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca50ada-1a9a-4455-be30-3b586ea92476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:06.198156 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca50ada-1a9a-4455-be30-3b586ea92476 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.198441 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca50ada-1a9a-4455-be30-3b586ea92476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.198719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca50ada-1a9a-4455-be30-3b586ea92476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.198991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1331/2680] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:52:06.199455 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca50ada-1a9a-4455-be30-3b586ea92476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:06.201037 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca50ada-1a9a-4455-be30-3b586ea92476 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:06.208981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f732e-d0ff-402a-9ab5-9e4747552087 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:06.210464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-517f732e-d0ff-402a-9ab5-9e4747552087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:06.214387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1350/2681] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:06.215797 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca50ada-1a9a-4455-be30-3b586ea92476 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:06.225719 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca50ada-1a9a-4455-be30-3b586ea92476 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:06.230959 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f90eb-0ec4-4126-8176-04c3aa479030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:06.231606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f90eb-0ec4-4126-8176-04c3aa479030 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.231888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f90eb-0ec4-4126-8176-04c3aa479030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.232169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f90eb-0ec4-4126-8176-04c3aa479030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.232878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f90eb-0ec4-4126-8176-04c3aa479030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:06.234496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f90eb-0ec4-4126-8176-04c3aa479030 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:06.245605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1351/2682] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:52:06.249954 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f90eb-0ec4-4126-8176-04c3aa479030 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:06.261138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f90eb-0ec4-4126-8176-04c3aa479030 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:06.265770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876bb48-a613-4159-8e3f-24994c6cc9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:06.266644 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876bb48-a613-4159-8e3f-24994c6cc9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.266983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876bb48-a613-4159-8e3f-24994c6cc9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.267220 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876bb48-a613-4159-8e3f-24994c6cc9b5 admin admin] PATH_INFO: `/v3/projects/983e4e7d6305486e905b46c097abd9cc/users/57897df0238a4e4b871d731b224e30f6/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.268725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f732e-d0ff-402a-9ab5-9e4747552087 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:06.273908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd567b-83bf-4e73-9d3e-67370700badb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:06.277869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd567b-83bf-4e73-9d3e-67370700badb service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.277869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd567b-83bf-4e73-9d3e-67370700badb service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.277869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd567b-83bf-4e73-9d3e-67370700badb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.283557 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876bb48-a613-4159-8e3f-24994c6cc9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983e4e7d6305486e905b46c097abd9cc, user_id=57897df0238a4e4b871d731b224e30f6, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:06.284998 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd567b-83bf-4e73-9d3e-67370700badb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:06.286493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd567b-83bf-4e73-9d3e-67370700badb service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:06.286616 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876bb48-a613-4159-8e3f-24994c6cc9b5 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:06.298675 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1332/2683] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:52:06.357043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1352/2684] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:06 2022] PUT /identity/v3/projects/983e4e7d6305486e905b46c097abd9cc/users/57897df0238a4e4b871d731b224e30f6/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:06.368750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2442c21-e9f2-4e0c-a9de-44538eabde35 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.369044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2442c21-e9f2-4e0c-a9de-44538eabde35 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.369634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2442c21-e9f2-4e0c-a9de-44538eabde35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.404222 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-f2442c21-e9f2-4e0c-a9de-44538eabde35 None None] MFA Rules not processed for user `57897df0238a4e4b871d731b224e30f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:06.405590 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f2442c21-e9f2-4e0c-a9de-44538eabde35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:06.479471 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1333/2685] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:06.484009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876bb48-a613-4159-8e3f-24994c6cc9b5 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:06.556649 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876bb48-a613-4159-8e3f-24994c6cc9b5 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:06.561386 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7b687-f9c9-41c7-ae5b-663672a0685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:06.562044 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7b687-f9c9-41c7-ae5b-663672a0685c admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.562351 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7b687-f9c9-41c7-ae5b-663672a0685c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.562623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7b687-f9c9-41c7-ae5b-663672a0685c admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.563349 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7b687-f9c9-41c7-ae5b-663672a0685c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:06.565262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7b687-f9c9-41c7-ae5b-663672a0685c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:06.659741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1353/2686] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:06 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:06.664116 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2442c21-e9f2-4e0c-a9de-44538eabde35 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:06.675517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2442c21-e9f2-4e0c-a9de-44538eabde35 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:06.746694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0f468-e521-4749-9b8b-360db01c95b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:06.747283 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0f468-e521-4749-9b8b-360db01c95b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.747568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0f468-e521-4749-9b8b-360db01c95b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.747884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0f468-e521-4749-9b8b-360db01c95b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.748804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0f468-e521-4749-9b8b-360db01c95b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:06.752118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0f468-e521-4749-9b8b-360db01c95b2 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:06.787956 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7b687-f9c9-41c7-ae5b-663672a0685c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:06.789771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-59f7b687-f9c9-41c7-ae5b-663672a0685c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:06.808978 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7b687-f9c9-41c7-ae5b-663672a0685c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:06.816010 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:06.817437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.817437 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.817667 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] PATH_INFO: `/v3/projects/9940fcecf9754f8e8cb0dac2188f1322` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.818771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9940fcecf9754f8e8cb0dac2188f1322)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:06.820936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:06.876125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1334/2687] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:06 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:06.877090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0f468-e521-4749-9b8b-360db01c95b2 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:06.887243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0f468-e521-4749-9b8b-360db01c95b2 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:06.892044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f28125-f341-4e36-8eaa-34cb68dd26b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:06.892701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f28125-f341-4e36-8eaa-34cb68dd26b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.892978 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f28125-f341-4e36-8eaa-34cb68dd26b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.893290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f28125-f341-4e36-8eaa-34cb68dd26b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.894049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f28125-f341-4e36-8eaa-34cb68dd26b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:06.896385 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f28125-f341-4e36-8eaa-34cb68dd26b0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:06.898493 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:06.906597 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1335/2688] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:52:06.911327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f28125-f341-4e36-8eaa-34cb68dd26b0 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:06.920705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f28125-f341-4e36-8eaa-34cb68dd26b0 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:06.925448 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61918ccf-da38-43b3-a7b3-cbc08cd15a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:06.926339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61918ccf-da38-43b3-a7b3-cbc08cd15a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:06.926616 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61918ccf-da38-43b3-a7b3-cbc08cd15a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:06.926890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61918ccf-da38-43b3-a7b3-cbc08cd15a59 admin admin] PATH_INFO: `/v3/projects/ce8e41ae712e4779a64ac424cb9ecdb4/users/145d852ef8c04f81a0a655cc16a5ff72/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:06.942128 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61918ccf-da38-43b3-a7b3-cbc08cd15a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8e41ae712e4779a64ac424cb9ecdb4, user_id=145d852ef8c04f81a0a655cc16a5ff72, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:06.945048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61918ccf-da38-43b3-a7b3-cbc08cd15a59 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:06.956535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9940fcecf9754f8e8cb0dac2188f1322', 'request_id': 'req-fc3dbb9e-6d59-4446-9ece-3819370c4739'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:07.026139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1336/2689] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:06 2022] PUT /identity/v3/projects/ce8e41ae712e4779a64ac424cb9ecdb4/users/145d852ef8c04f81a0a655cc16a5ff72/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:07.029971 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61918ccf-da38-43b3-a7b3-cbc08cd15a59 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.041509 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9940fcecf9754f8e8cb0dac2188f1322', 'request_id': 'req-fc3dbb9e-6d59-4446-9ece-3819370c4739'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:07.048105 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9940fcecf9754f8e8cb0dac2188f1322', 'request_id': 'req-fc3dbb9e-6d59-4446-9ece-3819370c4739'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:07.060260 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] The token cache is being invalidate because project 9940fcecf9754f8e8cb0dac2188f1322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:07.060549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:07.061244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-fc3dbb9e-6d59-4446-9ece-3819370c4739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:07.062840 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1354/2690] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:06 2022] DELETE /identity/v3/projects/9940fcecf9754f8e8cb0dac2188f1322 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:07.068662 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbfeec-b860-4cde-9108-2b9cadd198b1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.069036 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbfeec-b860-4cde-9108-2b9cadd198b1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.069363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbfeec-b860-4cde-9108-2b9cadd198b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.081741 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:07.090678 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-0bcbfeec-b860-4cde-9108-2b9cadd198b1 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:07.092014 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-0bcbfeec-b860-4cde-9108-2b9cadd198b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:07.102140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1355/2691] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:52:07.102776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61918ccf-da38-43b3-a7b3-cbc08cd15a59 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:07.107926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce4174-865b-4d9a-acc5-8978e5476d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:07.109071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce4174-865b-4d9a-acc5-8978e5476d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.109280 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce4174-865b-4d9a-acc5-8978e5476d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.109565 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce4174-865b-4d9a-acc5-8978e5476d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.110315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce4174-865b-4d9a-acc5-8978e5476d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:07.110819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced25bb-987c-452d-9050-8cc2624a7da7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.114035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced25bb-987c-452d-9050-8cc2624a7da7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.114206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce4174-865b-4d9a-acc5-8978e5476d68 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:07.115237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced25bb-987c-452d-9050-8cc2624a7da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.124190 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1337/2692] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:52:07.127898 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce4174-865b-4d9a-acc5-8978e5476d68 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.129954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e2ce4174-865b-4d9a-acc5-8978e5476d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:07.137363 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-4ced25bb-987c-452d-9050-8cc2624a7da7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:07.139422 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-4ced25bb-987c-452d-9050-8cc2624a7da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:07.142696 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce4174-865b-4d9a-acc5-8978e5476d68 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:07.148147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce1360-51f4-4fae-8dc0-4b2fd7202208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:07.148381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1356/2693] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:52:07.156596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1360-51f4-4fae-8dc0-4b2fd7202208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.156596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1360-51f4-4fae-8dc0-4b2fd7202208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.156596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1360-51f4-4fae-8dc0-4b2fd7202208 admin admin] PATH_INFO: `/v3/projects/ce8e41ae712e4779a64ac424cb9ecdb4/users/145d852ef8c04f81a0a655cc16a5ff72/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.156596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce1360-51f4-4fae-8dc0-4b2fd7202208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8e41ae712e4779a64ac424cb9ecdb4, user_id=145d852ef8c04f81a0a655cc16a5ff72, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:07.156596 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce1360-51f4-4fae-8dc0-4b2fd7202208 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:07.158909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced25bb-987c-452d-9050-8cc2624a7da7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.167949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1338/2694] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:07 2022] PUT /identity/v3/projects/ce8e41ae712e4779a64ac424cb9ecdb4/users/145d852ef8c04f81a0a655cc16a5ff72/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:07.169199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced25bb-987c-452d-9050-8cc2624a7da7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:07.178939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2ff84-5562-4a74-bfc3-68373e3cda10 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.179299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2ff84-5562-4a74-bfc3-68373e3cda10 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.179577 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2ff84-5562-4a74-bfc3-68373e3cda10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.212076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-b7f2ff84-5562-4a74-bfc3-68373e3cda10 None None] MFA Rules not processed for user `145d852ef8c04f81a0a655cc16a5ff72`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:07.213420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b7f2ff84-5562-4a74-bfc3-68373e3cda10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:07.234199 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45728223-67fc-4e98-85a1-44e44cb766d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:07.235148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45728223-67fc-4e98-85a1-44e44cb766d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.235436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45728223-67fc-4e98-85a1-44e44cb766d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.235712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45728223-67fc-4e98-85a1-44e44cb766d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.236636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45728223-67fc-4e98-85a1-44e44cb766d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:52:07.236951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45728223-67fc-4e98-85a1-44e44cb766d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:07.238694 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45728223-67fc-4e98-85a1-44e44cb766d2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:07.252432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1357/2695] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:52:07.257751 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45728223-67fc-4e98-85a1-44e44cb766d2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.268432 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45728223-67fc-4e98-85a1-44e44cb766d2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:07.275025 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8eb1b6-1b88-4fd3-b2d6-f1643cbb6a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:07.275697 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb1b6-1b88-4fd3-b2d6-f1643cbb6a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.275982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb1b6-1b88-4fd3-b2d6-f1643cbb6a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.276323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb1b6-1b88-4fd3-b2d6-f1643cbb6a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.277013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8eb1b6-1b88-4fd3-b2d6-f1643cbb6a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:07.278990 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8eb1b6-1b88-4fd3-b2d6-f1643cbb6a62 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:07.308509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1339/2696] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:52:07.320640 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2ff84-5562-4a74-bfc3-68373e3cda10 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.322455 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b7f2ff84-5562-4a74-bfc3-68373e3cda10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:07.374473 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:07.389722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2ff84-5562-4a74-bfc3-68373e3cda10 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:07.393457 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef16d2-1213-4139-8dc9-6ffaff4db24a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:07.394117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef16d2-1213-4139-8dc9-6ffaff4db24a service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.394402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef16d2-1213-4139-8dc9-6ffaff4db24a service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.394677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef16d2-1213-4139-8dc9-6ffaff4db24a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.407069 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef16d2-1213-4139-8dc9-6ffaff4db24a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:07.408543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef16d2-1213-4139-8dc9-6ffaff4db24a service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:07.413853 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1358/2697] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:07 2022] POST /identity/v3/projects => generated 389 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:52:07.418213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8eb1b6-1b88-4fd3-b2d6-f1643cbb6a62 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.425256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1340/2698] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:52:07.454386 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef16d2-1213-4139-8dc9-6ffaff4db24a service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.474320 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef16d2-1213-4139-8dc9-6ffaff4db24a service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:07.505094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8eb1b6-1b88-4fd3-b2d6-f1643cbb6a62 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:07.510236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d2929-22a8-411d-9765-6c2921953880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:07.510889 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d2929-22a8-411d-9765-6c2921953880 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.511184 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d2929-22a8-411d-9765-6c2921953880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.511552 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d2929-22a8-411d-9765-6c2921953880 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.512274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d2929-22a8-411d-9765-6c2921953880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:07.514431 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d2929-22a8-411d-9765-6c2921953880 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:07.549919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:07.550739 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.551049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.551352 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] PATH_INFO: `/v3/users/57897df0238a4e4b871d731b224e30f6` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.552571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] RBAC: Authorizing `identity:delete_user(user_id=57897df0238a4e4b871d731b224e30f6)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:07.554542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:07.614922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1359/2699] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:07 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:52:07.619090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d2929-22a8-411d-9765-6c2921953880 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.629104 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d2929-22a8-411d-9765-6c2921953880 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:07.634819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a5cc8-66c8-4971-9cdf-06dbc9c20619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:07.635482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a5cc8-66c8-4971-9cdf-06dbc9c20619 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.635779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a5cc8-66c8-4971-9cdf-06dbc9c20619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.636031 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a5cc8-66c8-4971-9cdf-06dbc9c20619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.636764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a5cc8-66c8-4971-9cdf-06dbc9c20619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:07.638394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a5cc8-66c8-4971-9cdf-06dbc9c20619 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:07.649138 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1360/2700] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:52:07.652973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a5cc8-66c8-4971-9cdf-06dbc9c20619 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.662742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a5cc8-66c8-4971-9cdf-06dbc9c20619 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:07.667610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334087b-5e0e-486c-a744-a483672fc5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:07.673819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334087b-5e0e-486c-a744-a483672fc5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.673819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334087b-5e0e-486c-a744-a483672fc5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.673819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334087b-5e0e-486c-a744-a483672fc5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.673819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334087b-5e0e-486c-a744-a483672fc5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:07.673819 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334087b-5e0e-486c-a744-a483672fc5e1 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:07.682513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1361/2701] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:52:07.686513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334087b-5e0e-486c-a744-a483672fc5e1 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.697306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334087b-5e0e-486c-a744-a483672fc5e1 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:07.702698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47894e76-3a99-43a1-8e22-42ac049414b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:07.703519 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47894e76-3a99-43a1-8e22-42ac049414b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.703519 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47894e76-3a99-43a1-8e22-42ac049414b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.703684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47894e76-3a99-43a1-8e22-42ac049414b4 admin admin] PATH_INFO: `/v3/projects/6968eb6fafce4e01a4626c6aabf09c15/users/4cb6ba13b1384feaa7005f7360d2f1bc/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.725615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47894e76-3a99-43a1-8e22-42ac049414b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6968eb6fafce4e01a4626c6aabf09c15, user_id=4cb6ba13b1384feaa7005f7360d2f1bc, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:07.731553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47894e76-3a99-43a1-8e22-42ac049414b4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:07.767667 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57897df0238a4e4b871d731b224e30f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:07.768567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57897df0238a4e4b871d731b224e30f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:07.781456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57897df0238a4e4b871d731b224e30f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:07.788900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57897df0238a4e4b871d731b224e30f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:07.836350 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1362/2702] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:07 2022] PUT /identity/v3/projects/6968eb6fafce4e01a4626c6aabf09c15/users/4cb6ba13b1384feaa7005f7360d2f1bc/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:07.847111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d578e6-9ae6-4def-9cf7-563e51a6f344 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.847426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d578e6-9ae6-4def-9cf7-563e51a6f344 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.847708 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d578e6-9ae6-4def-9cf7-563e51a6f344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.866730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:07.870169 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-b3b01e1c-804e-4335-b84a-5dbae78ff53e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57897df0238a4e4b871d731b224e30f6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:07.876864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-b0d578e6-9ae6-4def-9cf7-563e51a6f344 None None] MFA Rules not processed for user `4cb6ba13b1384feaa7005f7360d2f1bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:07.877145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1341/2703] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:07 2022] DELETE /identity/v3/users/57897df0238a4e4b871d731b224e30f6 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:07.878198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b0d578e6-9ae6-4def-9cf7-563e51a6f344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:07.893098 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1e87d-dcf8-40eb-a71c-f49babceaccf None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:07.894243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1e87d-dcf8-40eb-a71c-f49babceaccf None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:07.895636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1e87d-dcf8-40eb-a71c-f49babceaccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:07.924051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-a0b1e87d-dcf8-40eb-a71c-f49babceaccf None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:07.925310 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a0b1e87d-dcf8-40eb-a71c-f49babceaccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:07.955523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1363/2704] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:52:07.970416 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d578e6-9ae6-4def-9cf7-563e51a6f344 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:07.973587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b0d578e6-9ae6-4def-9cf7-563e51a6f344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:08.028893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1342/2705] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:08.040289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1e87d-dcf8-40eb-a71c-f49babceaccf None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:08.045315 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a0b1e87d-dcf8-40eb-a71c-f49babceaccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:08.052908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:08.065935 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d578e6-9ae6-4def-9cf7-563e51a6f344 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:08.069125 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ee3f2-5e3f-4519-8fbb-49d40d542fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:08.069891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ee3f2-5e3f-4519-8fbb-49d40d542fba service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:08.070174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ee3f2-5e3f-4519-8fbb-49d40d542fba service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:08.070446 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ee3f2-5e3f-4519-8fbb-49d40d542fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:08.079791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ee3f2-5e3f-4519-8fbb-49d40d542fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:08.081276 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ee3f2-5e3f-4519-8fbb-49d40d542fba service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:08.092730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1364/2706] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:08.113340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1e87d-dcf8-40eb-a71c-f49babceaccf None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:08.117945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c08aa1-729e-43bb-a86a-70a7453793f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:08.118587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c08aa1-729e-43bb-a86a-70a7453793f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:08.118866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c08aa1-729e-43bb-a86a-70a7453793f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:08.119158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c08aa1-729e-43bb-a86a-70a7453793f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:08.128517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c08aa1-729e-43bb-a86a-70a7453793f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:08.130429 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c08aa1-729e-43bb-a86a-70a7453793f4 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:08.140796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1343/2707] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:52:08.575454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ee3f2-5e3f-4519-8fbb-49d40d542fba service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:08.577256 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-585ee3f2-5e3f-4519-8fbb-49d40d542fba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:08.595147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ee3f2-5e3f-4519-8fbb-49d40d542fba service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:08.600408 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:08.601293 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:08.601589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:08.601899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] PATH_INFO: `/v3/projects/983e4e7d6305486e905b46c097abd9cc` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:08.606131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=983e4e7d6305486e905b46c097abd9cc)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:08.606131 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:08.729475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983e4e7d6305486e905b46c097abd9cc', 'request_id': 'req-5c967905-102f-48e5-ac64-641d66b18f35'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:08.795847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983e4e7d6305486e905b46c097abd9cc', 'request_id': 'req-5c967905-102f-48e5-ac64-641d66b18f35'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:08.811404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983e4e7d6305486e905b46c097abd9cc', 'request_id': 'req-5c967905-102f-48e5-ac64-641d66b18f35'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:08.831064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] The token cache is being invalidate because project 983e4e7d6305486e905b46c097abd9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:08.831323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:08.832093 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:08.836896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1365/2708] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:08 2022] DELETE /identity/v3/projects/983e4e7d6305486e905b46c097abd9cc => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:08.839014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c08aa1-729e-43bb-a86a-70a7453793f4 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:08.855420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c08aa1-729e-43bb-a86a-70a7453793f4 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:08.942893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:08.944344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:08.944613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:08.944886 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] PATH_INFO: `/v3/users/145d852ef8c04f81a0a655cc16a5ff72` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:08.946328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=145d852ef8c04f81a0a655cc16a5ff72)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:08.948257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:09.160011 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '145d852ef8c04f81a0a655cc16a5ff72'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:09.160782 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '145d852ef8c04f81a0a655cc16a5ff72'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:09.176765 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '145d852ef8c04f81a0a655cc16a5ff72'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:09.184546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '145d852ef8c04f81a0a655cc16a5ff72'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:09.247401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-626cd2d6-5980-43a7-b6a6-124865700e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '145d852ef8c04f81a0a655cc16a5ff72'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:09.253927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1344/2709] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:08 2022] DELETE /identity/v3/users/145d852ef8c04f81a0a655cc16a5ff72 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:09.786094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:09.787773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:09.807970 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c967905-102f-48e5-ac64-641d66b18f35 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:09.831847 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:09.874333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:09.875201 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:09.875485 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:09.875759 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] PATH_INFO: `/v3/projects/ce8e41ae712e4779a64ac424cb9ecdb4` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:09.877023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce8e41ae712e4779a64ac424cb9ecdb4)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:09.878858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:09.991713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce8e41ae712e4779a64ac424cb9ecdb4', 'request_id': 'req-745345a7-afa8-4add-9dcc-7e3bd3682899'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:10.070624 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce8e41ae712e4779a64ac424cb9ecdb4', 'request_id': 'req-745345a7-afa8-4add-9dcc-7e3bd3682899'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:10.076835 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce8e41ae712e4779a64ac424cb9ecdb4', 'request_id': 'req-745345a7-afa8-4add-9dcc-7e3bd3682899'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:10.089266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] The token cache is being invalidate because project ce8e41ae712e4779a64ac424cb9ecdb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:10.089558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:10.090210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-745345a7-afa8-4add-9dcc-7e3bd3682899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:10.091781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1366/2710] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:09 2022] DELETE /identity/v3/projects/ce8e41ae712e4779a64ac424cb9ecdb4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:10.096953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7501020-9fed-4d36-bd9f-2413668c0a14 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:10.097458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7501020-9fed-4d36-bd9f-2413668c0a14 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:10.097831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7501020-9fed-4d36-bd9f-2413668c0a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:10.117421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e7501020-9fed-4d36-bd9f-2413668c0a14 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:10.118632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e7501020-9fed-4d36-bd9f-2413668c0a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:10.139743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:10.180607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1345/2711] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:10.188413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb4962-7670-4c42-aa02-54eac11340b4 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:10.188716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb4962-7670-4c42-aa02-54eac11340b4 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:10.188994 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb4962-7670-4c42-aa02-54eac11340b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:10.207418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-53fb4962-7670-4c42-aa02-54eac11340b4 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:10.208620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-53fb4962-7670-4c42-aa02-54eac11340b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:10.216705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1367/2712] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:10.221568 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7501020-9fed-4d36-bd9f-2413668c0a14 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:10.232706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7501020-9fed-4d36-bd9f-2413668c0a14 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:10.238118 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c0bd0-c94e-4804-8d41-f9e952cf94cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:10.247510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c0bd0-c94e-4804-8d41-f9e952cf94cb admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:10.247510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c0bd0-c94e-4804-8d41-f9e952cf94cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:10.247510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c0bd0-c94e-4804-8d41-f9e952cf94cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:10.247510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c0bd0-c94e-4804-8d41-f9e952cf94cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:52:10.247510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c0bd0-c94e-4804-8d41-f9e952cf94cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:10.247510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c0bd0-c94e-4804-8d41-f9e952cf94cb admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:10.255822 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1346/2713] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:52:10.260264 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb4962-7670-4c42-aa02-54eac11340b4 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:10.271089 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb4962-7670-4c42-aa02-54eac11340b4 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:10.275894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90121488-4b18-4e32-a600-215f3aee7e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:10.276546 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90121488-4b18-4e32-a600-215f3aee7e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:10.276829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90121488-4b18-4e32-a600-215f3aee7e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:10.277171 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90121488-4b18-4e32-a600-215f3aee7e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:10.277870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90121488-4b18-4e32-a600-215f3aee7e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:10.279791 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90121488-4b18-4e32-a600-215f3aee7e02 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:10.391544 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1368/2714] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:10 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:52:10.393147 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c0bd0-c94e-4804-8d41-f9e952cf94cb admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:10.404226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c0bd0-c94e-4804-8d41-f9e952cf94cb admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:10.463599 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d6934-4ff5-4b72-8f7a-4df7b8f4bfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:10.464406 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d6934-4ff5-4b72-8f7a-4df7b8f4bfd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:10.464692 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d6934-4ff5-4b72-8f7a-4df7b8f4bfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:10.464968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d6934-4ff5-4b72-8f7a-4df7b8f4bfd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:10.465703 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d6934-4ff5-4b72-8f7a-4df7b8f4bfd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:10.471178 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d6934-4ff5-4b72-8f7a-4df7b8f4bfd2 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:10.555649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1347/2715] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:10 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:10.559893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90121488-4b18-4e32-a600-215f3aee7e02 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:10.571343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90121488-4b18-4e32-a600-215f3aee7e02 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:10.575927 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4960cf6-3dac-4cdd-95fe-9954bfa9ce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:10.576581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4960cf6-3dac-4cdd-95fe-9954bfa9ce34 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:10.576854 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4960cf6-3dac-4cdd-95fe-9954bfa9ce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:10.577122 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4960cf6-3dac-4cdd-95fe-9954bfa9ce34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:10.577877 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4960cf6-3dac-4cdd-95fe-9954bfa9ce34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:10.579433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4960cf6-3dac-4cdd-95fe-9954bfa9ce34 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:10.589801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1369/2716] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:10.594000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d6934-4ff5-4b72-8f7a-4df7b8f4bfd2 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:10.604119 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d6934-4ff5-4b72-8f7a-4df7b8f4bfd2 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:10.608800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4883dcc-668f-49fc-a7a6-82ce8a4b1c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:10.609500 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4883dcc-668f-49fc-a7a6-82ce8a4b1c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:10.609783 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4883dcc-668f-49fc-a7a6-82ce8a4b1c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:10.610058 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4883dcc-668f-49fc-a7a6-82ce8a4b1c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:10.610783 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4883dcc-668f-49fc-a7a6-82ce8a4b1c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:10.612383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4883dcc-668f-49fc-a7a6-82ce8a4b1c9b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:10.626391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1348/2717] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:10.627522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4960cf6-3dac-4cdd-95fe-9954bfa9ce34 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:10.638718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4960cf6-3dac-4cdd-95fe-9954bfa9ce34 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:10.643621 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef953a4-56b5-4f5a-b0e5-867caab7981a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:10.644610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef953a4-56b5-4f5a-b0e5-867caab7981a admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:10.644974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef953a4-56b5-4f5a-b0e5-867caab7981a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:10.645717 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef953a4-56b5-4f5a-b0e5-867caab7981a admin admin] PATH_INFO: `/v3/projects/e5d5d10d293e42ed9bf235340bf7cc53/users/0de97d8f34434c3d9d23f57bae349805/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:10.662361 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef953a4-56b5-4f5a-b0e5-867caab7981a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d5d10d293e42ed9bf235340bf7cc53, user_id=0de97d8f34434c3d9d23f57bae349805, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:10.665017 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef953a4-56b5-4f5a-b0e5-867caab7981a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:10.720868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1370/2718] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:10 2022] PUT /identity/v3/projects/e5d5d10d293e42ed9bf235340bf7cc53/users/0de97d8f34434c3d9d23f57bae349805/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:10.734984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338b5b7-189f-44fd-a520-14487e3cc130 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:10.734984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338b5b7-189f-44fd-a520-14487e3cc130 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:10.734984 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338b5b7-189f-44fd-a520-14487e3cc130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:10.761603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-3338b5b7-189f-44fd-a520-14487e3cc130 None None] MFA Rules not processed for user `0de97d8f34434c3d9d23f57bae349805`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:10.762865 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3338b5b7-189f-44fd-a520-14487e3cc130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:10.832690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1349/2719] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:10.837821 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef953a4-56b5-4f5a-b0e5-867caab7981a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:10.911160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:10.923371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef953a4-56b5-4f5a-b0e5-867caab7981a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:10.928430 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd628a-9892-4db6-94b4-9e45228ff317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:10.929090 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd628a-9892-4db6-94b4-9e45228ff317 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:10.929417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd628a-9892-4db6-94b4-9e45228ff317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:10.929682 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd628a-9892-4db6-94b4-9e45228ff317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:10.930398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd628a-9892-4db6-94b4-9e45228ff317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:10.932355 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd628a-9892-4db6-94b4-9e45228ff317 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:11.044887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1371/2720] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:10 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:52:11.049005 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338b5b7-189f-44fd-a520-14487e3cc130 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:11.060980 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338b5b7-189f-44fd-a520-14487e3cc130 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:11.079866 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:11.118694 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac309f91-0b63-4592-a3b8-ba1cf51955d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:11.119895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac309f91-0b63-4592-a3b8-ba1cf51955d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:11.119895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac309f91-0b63-4592-a3b8-ba1cf51955d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:11.120205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac309f91-0b63-4592-a3b8-ba1cf51955d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:11.120762 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac309f91-0b63-4592-a3b8-ba1cf51955d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:11.122745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac309f91-0b63-4592-a3b8-ba1cf51955d4 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:11.249097 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1350/2721] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:11 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:11.252920 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd628a-9892-4db6-94b4-9e45228ff317 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:11.263882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd628a-9892-4db6-94b4-9e45228ff317 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:11.268424 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e65a3-ca9f-4188-b37a-c8f631e1a17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:11.269100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e65a3-ca9f-4188-b37a-c8f631e1a17d admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:11.273737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e65a3-ca9f-4188-b37a-c8f631e1a17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:11.273737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e65a3-ca9f-4188-b37a-c8f631e1a17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:11.273737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e65a3-ca9f-4188-b37a-c8f631e1a17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:11.273737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e65a3-ca9f-4188-b37a-c8f631e1a17d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:11.283651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1372/2722] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:11.287852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac309f91-0b63-4592-a3b8-ba1cf51955d4 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:11.297953 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac309f91-0b63-4592-a3b8-ba1cf51955d4 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:11.303226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620199f-075a-4b69-b002-0a417cbdf00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:11.303882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620199f-075a-4b69-b002-0a417cbdf00b admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:11.304167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620199f-075a-4b69-b002-0a417cbdf00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:11.304446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620199f-075a-4b69-b002-0a417cbdf00b admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:11.305203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620199f-075a-4b69-b002-0a417cbdf00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:11.306804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620199f-075a-4b69-b002-0a417cbdf00b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:11.316780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1351/2723] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:11.321535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e65a3-ca9f-4188-b37a-c8f631e1a17d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:11.331344 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e65a3-ca9f-4188-b37a-c8f631e1a17d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:11.336951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cba51-d8e0-41da-ad78-72a18f5c62f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:11.336951 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cba51-d8e0-41da-ad78-72a18f5c62f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:11.337203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cba51-d8e0-41da-ad78-72a18f5c62f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:11.337325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cba51-d8e0-41da-ad78-72a18f5c62f0 admin admin] PATH_INFO: `/v3/projects/6372e09e2c8d4df8af6681b74b4592bd/users/41b6b61a4d8c4f63aa29b783fe15790e/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:11.351548 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4cba51-d8e0-41da-ad78-72a18f5c62f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6372e09e2c8d4df8af6681b74b4592bd, user_id=41b6b61a4d8c4f63aa29b783fe15790e, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:11.354200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4cba51-d8e0-41da-ad78-72a18f5c62f0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:11.435896 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1373/2724] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:11 2022] PUT /identity/v3/projects/6372e09e2c8d4df8af6681b74b4592bd/users/41b6b61a4d8c4f63aa29b783fe15790e/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:11.501532 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be7257-34f1-4dbf-9c60-06cf9acce597 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:11.501761 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be7257-34f1-4dbf-9c60-06cf9acce597 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:11.502018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be7257-34f1-4dbf-9c60-06cf9acce597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:11.530061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-28be7257-34f1-4dbf-9c60-06cf9acce597 None None] MFA Rules not processed for user `41b6b61a4d8c4f63aa29b783fe15790e`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:11.531297 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-28be7257-34f1-4dbf-9c60-06cf9acce597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:11.597366 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1352/2725] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:11.601795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cba51-d8e0-41da-ad78-72a18f5c62f0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:11.663168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cba51-d8e0-41da-ad78-72a18f5c62f0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:11.667783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7702547-d050-4613-8f2d-ea7a4672bad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:11.668462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7702547-d050-4613-8f2d-ea7a4672bad7 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:11.668778 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7702547-d050-4613-8f2d-ea7a4672bad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:11.668985 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7702547-d050-4613-8f2d-ea7a4672bad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:11.669813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7702547-d050-4613-8f2d-ea7a4672bad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:11.671697 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7702547-d050-4613-8f2d-ea7a4672bad7 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:11.764043 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1374/2726] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:11 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:52:11.768051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be7257-34f1-4dbf-9c60-06cf9acce597 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:11.779763 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be7257-34f1-4dbf-9c60-06cf9acce597 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:11.837629 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb20b8-8ce5-472a-9700-bc7d5c96537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:11.838314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb20b8-8ce5-472a-9700-bc7d5c96537e admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:11.838657 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb20b8-8ce5-472a-9700-bc7d5c96537e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:11.838875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb20b8-8ce5-472a-9700-bc7d5c96537e admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:11.839586 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb20b8-8ce5-472a-9700-bc7d5c96537e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:11.841542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb20b8-8ce5-472a-9700-bc7d5c96537e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:11.941237 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1353/2727] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:11 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:11.945915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7702547-d050-4613-8f2d-ea7a4672bad7 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:11.957948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7702547-d050-4613-8f2d-ea7a4672bad7 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:11.963964 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af3278-cb62-4662-939a-4df5b61b5555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:11.964642 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af3278-cb62-4662-939a-4df5b61b5555 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:11.964926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af3278-cb62-4662-939a-4df5b61b5555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:11.965236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af3278-cb62-4662-939a-4df5b61b5555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:11.966033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af3278-cb62-4662-939a-4df5b61b5555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:11.967634 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af3278-cb62-4662-939a-4df5b61b5555 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:11.980216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1375/2728] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:11.984699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb20b8-8ce5-472a-9700-bc7d5c96537e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:11.998112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb20b8-8ce5-472a-9700-bc7d5c96537e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:12.002935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651bbd4-fd65-4f8e-9fac-51fb00015bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:12.003856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651bbd4-fd65-4f8e-9fac-51fb00015bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:12.004141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651bbd4-fd65-4f8e-9fac-51fb00015bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:12.004426 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651bbd4-fd65-4f8e-9fac-51fb00015bb1 admin admin] PATH_INFO: `/v3/projects/ecb78846b3294d56b99b2d120991c805/users/185c4615110349949485578df52e0a2d/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:12.023887 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651bbd4-fd65-4f8e-9fac-51fb00015bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb78846b3294d56b99b2d120991c805, user_id=185c4615110349949485578df52e0a2d, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:12.026806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651bbd4-fd65-4f8e-9fac-51fb00015bb1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:12.030332 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:12.104804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1354/2729] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:11 2022] PUT /identity/v3/projects/ecb78846b3294d56b99b2d120991c805/users/185c4615110349949485578df52e0a2d/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:12.109113 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af3278-cb62-4662-939a-4df5b61b5555 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:12.120200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af3278-cb62-4662-939a-4df5b61b5555 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:12.185721 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f854266-3b32-412e-8146-3f8f7ebde671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:12.186398 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f854266-3b32-412e-8146-3f8f7ebde671 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:12.186682 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f854266-3b32-412e-8146-3f8f7ebde671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:12.186975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f854266-3b32-412e-8146-3f8f7ebde671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:12.187690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f854266-3b32-412e-8146-3f8f7ebde671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:12.189295 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f854266-3b32-412e-8146-3f8f7ebde671 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:12.199939 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1376/2730] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:12.209672 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651bbd4-fd65-4f8e-9fac-51fb00015bb1 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:12.221211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651bbd4-fd65-4f8e-9fac-51fb00015bb1 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:12.226535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261fa05-c059-4719-9bee-c4ebd6825b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:12.227460 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261fa05-c059-4719-9bee-c4ebd6825b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:12.227743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261fa05-c059-4719-9bee-c4ebd6825b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:12.228090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261fa05-c059-4719-9bee-c4ebd6825b71 admin admin] PATH_INFO: `/v3/projects/ecb78846b3294d56b99b2d120991c805/users/185c4615110349949485578df52e0a2d/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:12.230112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261fa05-c059-4719-9bee-c4ebd6825b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb78846b3294d56b99b2d120991c805, user_id=185c4615110349949485578df52e0a2d, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:12.232784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261fa05-c059-4719-9bee-c4ebd6825b71 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:12.244364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1355/2731] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:12 2022] PUT /identity/v3/projects/ecb78846b3294d56b99b2d120991c805/users/185c4615110349949485578df52e0a2d/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:12.254898 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7031e-b2db-4333-9221-71d586ec3b8f None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:12.255200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7031e-b2db-4333-9221-71d586ec3b8f None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:12.255467 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7031e-b2db-4333-9221-71d586ec3b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:12.277796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:12.285865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-8ec7031e-b2db-4333-9221-71d586ec3b8f None None] MFA Rules not processed for user `185c4615110349949485578df52e0a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:12.291973 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8ec7031e-b2db-4333-9221-71d586ec3b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:12.358319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1377/2732] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:12.370941 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261fa05-c059-4719-9bee-c4ebd6825b71 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:12.372664 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e261fa05-c059-4719-9bee-c4ebd6825b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:12.432715 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261fa05-c059-4719-9bee-c4ebd6825b71 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:12.436552 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b7c13-1ad2-412e-8dee-7302f7adcf28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:12.437296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b7c13-1ad2-412e-8dee-7302f7adcf28 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:12.437579 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b7c13-1ad2-412e-8dee-7302f7adcf28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:12.441096 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b7c13-1ad2-412e-8dee-7302f7adcf28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:12.447841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b7c13-1ad2-412e-8dee-7302f7adcf28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:12.449309 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b7c13-1ad2-412e-8dee-7302f7adcf28 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:12.459171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1356/2733] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:12.535861 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7031e-b2db-4333-9221-71d586ec3b8f None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:12.537386 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8ec7031e-b2db-4333-9221-71d586ec3b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:12.549251 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7031e-b2db-4333-9221-71d586ec3b8f None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:12.552760 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27c73c-0a9a-4719-b38e-ed4a2f40a390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:12.553423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27c73c-0a9a-4719-b38e-ed4a2f40a390 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:12.553706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27c73c-0a9a-4719-b38e-ed4a2f40a390 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:12.553972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27c73c-0a9a-4719-b38e-ed4a2f40a390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:12.610296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27c73c-0a9a-4719-b38e-ed4a2f40a390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:12.611727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27c73c-0a9a-4719-b38e-ed4a2f40a390 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:12.620894 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1378/2734] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:14.985737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9e7d8-7a58-4f7f-8330-ed414a669b2b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:14.988422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9e7d8-7a58-4f7f-8330-ed414a669b2b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:14.988422 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9e7d8-7a58-4f7f-8330-ed414a669b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:15.012749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-cba9e7d8-7a58-4f7f-8330-ed414a669b2b None None] MFA Rules not processed for user `1a71cad9c9114c5cb0bf8b4fe982e4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:15.017301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cba9e7d8-7a58-4f7f-8330-ed414a669b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:15.072188 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1357/2735] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:22.616730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=57 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:25.065307 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=58,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:29.600495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27c73c-0a9a-4719-b38e-ed4a2f40a390 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:29.672376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27c73c-0a9a-4719-b38e-ed4a2f40a390 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:29.678480 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:29.679490 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:29.679880 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:29.680155 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] PATH_INFO: `/v3/users/0de97d8f34434c3d9d23f57bae349805` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:29.681496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0de97d8f34434c3d9d23f57bae349805)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:29.684475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:29.956415 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0de97d8f34434c3d9d23f57bae349805'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:29.965803 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0de97d8f34434c3d9d23f57bae349805'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:29.998545 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0de97d8f34434c3d9d23f57bae349805'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:30.110274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0de97d8f34434c3d9d23f57bae349805'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:30.111119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0de97d8f34434c3d9d23f57bae349805'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:30.125513 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1379/2736] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:29 2022] DELETE /identity/v3/users/0de97d8f34434c3d9d23f57bae349805 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:30.129809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544908d-3115-4f3b-af84-fe14e8903a50 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:30.130477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544908d-3115-4f3b-af84-fe14e8903a50 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:30.130477 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544908d-3115-4f3b-af84-fe14e8903a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:30.155394 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-2544908d-3115-4f3b-af84-fe14e8903a50 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:30.156857 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2544908d-3115-4f3b-af84-fe14e8903a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:30.242377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1358/2737] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:30.256472 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:30.258495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:30.401244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59235b0-3194-48fe-9a54-5f05d0861140 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:30.401244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988aa7a-9419-4662-b5aa-96d412e0b89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:30.401244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988aa7a-9419-4662-b5aa-96d412e0b89a service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:30.401244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988aa7a-9419-4662-b5aa-96d412e0b89a service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:30.401865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988aa7a-9419-4662-b5aa-96d412e0b89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:30.416449 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988aa7a-9419-4662-b5aa-96d412e0b89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:30.419163 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988aa7a-9419-4662-b5aa-96d412e0b89a service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:30.436501 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1380/2738] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:31.026153 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544908d-3115-4f3b-af84-fe14e8903a50 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:31.027253 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-2544908d-3115-4f3b-af84-fe14e8903a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:31.039542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544908d-3115-4f3b-af84-fe14e8903a50 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:31.044949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:31.046123 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:31.046464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:31.046741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] PATH_INFO: `/v3/projects/e5d5d10d293e42ed9bf235340bf7cc53` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:31.048838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d5d10d293e42ed9bf235340bf7cc53)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:31.052024 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:31.189939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d5d10d293e42ed9bf235340bf7cc53', 'request_id': 'req-fc76d569-b07b-4926-938f-15d1a298cb35'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:31.196764 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d5d10d293e42ed9bf235340bf7cc53', 'request_id': 'req-fc76d569-b07b-4926-938f-15d1a298cb35'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:31.202930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d5d10d293e42ed9bf235340bf7cc53', 'request_id': 'req-fc76d569-b07b-4926-938f-15d1a298cb35'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:31.266443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] The token cache is being invalidate because project e5d5d10d293e42ed9bf235340bf7cc53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:31.266853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:31.267699 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:31.270993 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1359/2739] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:31 2022] DELETE /identity/v3/projects/e5d5d10d293e42ed9bf235340bf7cc53 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:31.275529 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988aa7a-9419-4662-b5aa-96d412e0b89a service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:31.280548 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:31.296266 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988aa7a-9419-4662-b5aa-96d412e0b89a service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:31.344278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:31.345232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:31.345517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:31.345792 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] PATH_INFO: `/v3/users/41b6b61a4d8c4f63aa29b783fe15790e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:31.346971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b6b61a4d8c4f63aa29b783fe15790e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:31.348895 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:31.500766 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b6b61a4d8c4f63aa29b783fe15790e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:31.507665 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b6b61a4d8c4f63aa29b783fe15790e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:31.520303 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b6b61a4d8c4f63aa29b783fe15790e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:31.578288 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b6b61a4d8c4f63aa29b783fe15790e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:31.579870 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b6b61a4d8c4f63aa29b783fe15790e'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:31.588356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1381/2740] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:31 2022] DELETE /identity/v3/users/41b6b61a4d8c4f63aa29b783fe15790e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:31.752353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:31.753816 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:31.767391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76d569-b07b-4926-938f-15d1a298cb35 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:31.837281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:31.838170 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:31.838447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:31.838721 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] PATH_INFO: `/v3/users/0b2f6947d03243b58caf3bc9243ca9bc` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:31.840798 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b2f6947d03243b58caf3bc9243ca9bc)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:31.842804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:31.933541 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:32.020548 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:32.022060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:32.038381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6586acf-e997-421d-96c1-323d33e34653 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:32.043608 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:32.045094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:32.045538 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:32.045815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] PATH_INFO: `/v3/projects/6372e09e2c8d4df8af6681b74b4592bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:32.047243 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6372e09e2c8d4df8af6681b74b4592bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:32.049418 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:32.050660 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b2f6947d03243b58caf3bc9243ca9bc'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.051197 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b2f6947d03243b58caf3bc9243ca9bc'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.063581 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b2f6947d03243b58caf3bc9243ca9bc'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.070561 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b2f6947d03243b58caf3bc9243ca9bc'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.139020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5be61b50-427b-4c51-a8a6-14f25636da15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b2f6947d03243b58caf3bc9243ca9bc'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.147018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1360/2741] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:31 2022] DELETE /identity/v3/users/0b2f6947d03243b58caf3bc9243ca9bc => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:32.151499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2f99d-1513-4af3-abab-fb8addc538d2 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:32.151779 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2f99d-1513-4af3-abab-fb8addc538d2 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:32.152051 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2f99d-1513-4af3-abab-fb8addc538d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:32.161274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6372e09e2c8d4df8af6681b74b4592bd', 'request_id': 'req-ef2d316f-38a6-4599-a800-bfbd1b47be1d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.171505 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-b5b2f99d-1513-4af3-abab-fb8addc538d2 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:32.172800 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b5b2f99d-1513-4af3-abab-fb8addc538d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:32.231509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1361/2742] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:32.241363 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2f99d-1513-4af3-abab-fb8addc538d2 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:32.242709 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-b5b2f99d-1513-4af3-abab-fb8addc538d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:32.258198 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6372e09e2c8d4df8af6681b74b4592bd', 'request_id': 'req-ef2d316f-38a6-4599-a800-bfbd1b47be1d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.268058 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6372e09e2c8d4df8af6681b74b4592bd', 'request_id': 'req-ef2d316f-38a6-4599-a800-bfbd1b47be1d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.277052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] The token cache is being invalidate because project 6372e09e2c8d4df8af6681b74b4592bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:32.277705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.278022 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.279762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1382/2743] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:32 2022] DELETE /identity/v3/projects/6372e09e2c8d4df8af6681b74b4592bd => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:32.283341 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:32.284743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:32.304384 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2f99d-1513-4af3-abab-fb8addc538d2 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:32.308817 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e0053-98c3-4173-ba0b-99dcf3dcbd36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:32.309554 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e0053-98c3-4173-ba0b-99dcf3dcbd36 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:32.309870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e0053-98c3-4173-ba0b-99dcf3dcbd36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:32.310150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e0053-98c3-4173-ba0b-99dcf3dcbd36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:32.312000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f84e0053-98c3-4173-ba0b-99dcf3dcbd36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:32.329888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e0053-98c3-4173-ba0b-99dcf3dcbd36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:32.331431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e0053-98c3-4173-ba0b-99dcf3dcbd36 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:32.342056 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1362/2744] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:52:32.362607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d316f-38a6-4599-a800-bfbd1b47be1d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:32.374773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:32.375801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:32.376193 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:32.376562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] PATH_INFO: `/v3/users/185c4615110349949485578df52e0a2d` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:32.378160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=185c4615110349949485578df52e0a2d)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:32.380549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:32.594425 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '185c4615110349949485578df52e0a2d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.603566 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '185c4615110349949485578df52e0a2d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.624087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '185c4615110349949485578df52e0a2d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.677661 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '185c4615110349949485578df52e0a2d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.678371 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '185c4615110349949485578df52e0a2d'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:32.689726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1383/2745] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:32 2022] DELETE /identity/v3/users/185c4615110349949485578df52e0a2d => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:32.749639 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:32.791358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e0053-98c3-4173-ba0b-99dcf3dcbd36 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:32.793844 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f84e0053-98c3-4173-ba0b-99dcf3dcbd36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:32.823518 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e0053-98c3-4173-ba0b-99dcf3dcbd36 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:32.886017 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:32.887515 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:32.887787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:32.888061 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] PATH_INFO: `/v3/projects/3de56edd4b0140b692b4e4f62deb5bd4` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:32.889285 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3de56edd4b0140b692b4e4f62deb5bd4)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:32.891172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:33.039517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3de56edd4b0140b692b4e4f62deb5bd4', 'request_id': 'req-aa4e395c-4590-46b2-a081-ef0b4e78ba34'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.046251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3de56edd4b0140b692b4e4f62deb5bd4', 'request_id': 'req-aa4e395c-4590-46b2-a081-ef0b4e78ba34'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.052447 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3de56edd4b0140b692b4e4f62deb5bd4', 'request_id': 'req-aa4e395c-4590-46b2-a081-ef0b4e78ba34'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.149991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] The token cache is being invalidate because project 3de56edd4b0140b692b4e4f62deb5bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:33.150282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.150944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-aa4e395c-4590-46b2-a081-ef0b4e78ba34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.152772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:33.154222 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:33.156349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1363/2746] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:32 2022] DELETE /identity/v3/projects/3de56edd4b0140b692b4e4f62deb5bd4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:33.163606 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a9232-7928-4b74-9db5-b9f9205062c9 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:33.163943 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a9232-7928-4b74-9db5-b9f9205062c9 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:33.164180 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a9232-7928-4b74-9db5-b9f9205062c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:33.168589 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ca8e6-1399-497e-aeab-43290bfc19d0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:33.187555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c34a9232-7928-4b74-9db5-b9f9205062c9 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:33.188937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c34a9232-7928-4b74-9db5-b9f9205062c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:33.238397 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:33.239333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:33.239611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:33.239887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] PATH_INFO: `/v3/projects/ecb78846b3294d56b99b2d120991c805` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:33.241205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb78846b3294d56b99b2d120991c805)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:33.243108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:33.253295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1364/2747] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:33.259916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcf824-e63d-42c1-8115-4c3cd25e7fbc None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:33.260207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcf824-e63d-42c1-8115-4c3cd25e7fbc None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:33.260516 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcf824-e63d-42c1-8115-4c3cd25e7fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:33.280949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-8adcf824-e63d-42c1-8115-4c3cd25e7fbc None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:33.283848 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8adcf824-e63d-42c1-8115-4c3cd25e7fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:33.291291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1365/2748] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:33.296074 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcf824-e63d-42c1-8115-4c3cd25e7fbc None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:33.305609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcf824-e63d-42c1-8115-4c3cd25e7fbc None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:33.311027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31b0a0-5840-436e-b579-ff342a6dcdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:33.311690 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31b0a0-5840-436e-b579-ff342a6dcdfe admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:33.311975 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31b0a0-5840-436e-b579-ff342a6dcdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:33.312255 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31b0a0-5840-436e-b579-ff342a6dcdfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:33.313099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31b0a0-5840-436e-b579-ff342a6dcdfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:52:33.313439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31b0a0-5840-436e-b579-ff342a6dcdfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:33.315132 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31b0a0-5840-436e-b579-ff342a6dcdfe admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:33.327158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1366/2749] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:52:33.331636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31b0a0-5840-436e-b579-ff342a6dcdfe admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:33.341519 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31b0a0-5840-436e-b579-ff342a6dcdfe admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:33.346318 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a32a27-68ec-45e8-9082-35d353f56cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:33.346957 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a32a27-68ec-45e8-9082-35d353f56cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:33.347239 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a32a27-68ec-45e8-9082-35d353f56cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:33.347514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a32a27-68ec-45e8-9082-35d353f56cd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:33.348256 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a32a27-68ec-45e8-9082-35d353f56cd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:33.350191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a32a27-68ec-45e8-9082-35d353f56cd2 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:33.367225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb78846b3294d56b99b2d120991c805', 'request_id': 'req-11860198-227d-4f65-96a8-90ebdbdc04cf'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.384754 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:33.471417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1367/2750] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:33 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:52:33.472364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a32a27-68ec-45e8-9082-35d353f56cd2 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:33.474004 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-69a32a27-68ec-45e8-9082-35d353f56cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:33.515215 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb78846b3294d56b99b2d120991c805', 'request_id': 'req-11860198-227d-4f65-96a8-90ebdbdc04cf'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.521881 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb78846b3294d56b99b2d120991c805', 'request_id': 'req-11860198-227d-4f65-96a8-90ebdbdc04cf'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.533386 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] The token cache is being invalidate because project ecb78846b3294d56b99b2d120991c805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:33.533677 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.534438 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.535972 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1384/2751] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:33 2022] DELETE /identity/v3/projects/ecb78846b3294d56b99b2d120991c805 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:33.536837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:33.538151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:33.543705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a32a27-68ec-45e8-9082-35d353f56cd2 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:33.548656 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:33.549585 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:33.549854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:33.550634 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] PATH_INFO: `/v3/users/4cb6ba13b1384feaa7005f7360d2f1bc` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:33.551342 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb6ba13b1384feaa7005f7360d2f1bc)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:33.553301 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:33.600960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11860198-227d-4f65-96a8-90ebdbdc04cf admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:33.606352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08761d67-f718-45c1-b902-7e409fcbaa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:33.606837 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08761d67-f718-45c1-b902-7e409fcbaa26 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:33.607560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08761d67-f718-45c1-b902-7e409fcbaa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:33.607560 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08761d67-f718-45c1-b902-7e409fcbaa26 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:33.608052 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08761d67-f718-45c1-b902-7e409fcbaa26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:33.609991 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08761d67-f718-45c1-b902-7e409fcbaa26 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:33.818211 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1385/2752] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:33 2022] POST /identity/v3/users => generated 282 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:33.819687 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f28034-95eb-4450-ac98-2aa03c35986b None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:33.819977 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f28034-95eb-4450-ac98-2aa03c35986b None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:33.820255 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f28034-95eb-4450-ac98-2aa03c35986b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:33.838133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-10f28034-95eb-4450-ac98-2aa03c35986b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:33.839869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-10f28034-95eb-4450-ac98-2aa03c35986b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:33.848456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1386/2753] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:33.857382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f28034-95eb-4450-ac98-2aa03c35986b None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:33.857603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb6ba13b1384feaa7005f7360d2f1bc'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.857603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb6ba13b1384feaa7005f7360d2f1bc'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.866021 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb6ba13b1384feaa7005f7360d2f1bc'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.867225 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f28034-95eb-4450-ac98-2aa03c35986b None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:33.871950 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b339b14-ad6e-4ada-b6ef-c6bff2bb609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:33.872623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b339b14-ad6e-4ada-b6ef-c6bff2bb609a admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:33.872908 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b339b14-ad6e-4ada-b6ef-c6bff2bb609a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:33.873283 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb6ba13b1384feaa7005f7360d2f1bc'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.873402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b339b14-ad6e-4ada-b6ef-c6bff2bb609a admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:33.874164 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b339b14-ad6e-4ada-b6ef-c6bff2bb609a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:33.875771 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b339b14-ad6e-4ada-b6ef-c6bff2bb609a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:33.885962 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1387/2754] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:33.887173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921efd-2be9-4f20-a043-b43697c1fd9d None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:33.887461 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921efd-2be9-4f20-a043-b43697c1fd9d None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:33.887737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3921efd-2be9-4f20-a043-b43697c1fd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:33.906902 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f3921efd-2be9-4f20-a043-b43697c1fd9d None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:33.908316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f3921efd-2be9-4f20-a043-b43697c1fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:33.917210 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1388/2755] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:33.918703 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3921efd-2be9-4f20-a043-b43697c1fd9d None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:33.920142 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f3921efd-2be9-4f20-a043-b43697c1fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:33.925786 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb6ba13b1384feaa7005f7360d2f1bc'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:33.932550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1368/2756] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:33 2022] DELETE /identity/v3/users/4cb6ba13b1384feaa7005f7360d2f1bc => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:33.933725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:33.934631 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3921efd-2be9-4f20-a043-b43697c1fd9d None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:33.973497 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:34.002259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa94b18-2e34-4fce-b447-c6a199583ad0 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.008010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f96dd-73b3-4ccb-9c31-e91cb99f0480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.008234 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c52a26-2567-4d2f-83c7-eeaeac307563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.008679 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f96dd-73b3-4ccb-9c31-e91cb99f0480 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.008868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c52a26-2567-4d2f-83c7-eeaeac307563 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.008991 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f96dd-73b3-4ccb-9c31-e91cb99f0480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.009150 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c52a26-2567-4d2f-83c7-eeaeac307563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.009279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f96dd-73b3-4ccb-9c31-e91cb99f0480 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.009454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c52a26-2567-4d2f-83c7-eeaeac307563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.010159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f96dd-73b3-4ccb-9c31-e91cb99f0480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:52:34.010353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f96dd-73b3-4ccb-9c31-e91cb99f0480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.011823 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c52a26-2567-4d2f-83c7-eeaeac307563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.012472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f96dd-73b3-4ccb-9c31-e91cb99f0480 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.013699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c52a26-2567-4d2f-83c7-eeaeac307563 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.025105 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1369/2757] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:52:34.029726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1389/2758] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:34.029908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5eebb-7c0b-4301-9be2-2effa25c9991 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.029908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5eebb-7c0b-4301-9be2-2effa25c9991 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.029908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5eebb-7c0b-4301-9be2-2effa25c9991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.030464 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c52a26-2567-4d2f-83c7-eeaeac307563 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:34.042403 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c52a26-2567-4d2f-83c7-eeaeac307563 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.047423 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9174d-23ac-41d0-bdfa-872e3ab3314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.047730 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:34.048364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9174d-23ac-41d0-bdfa-872e3ab3314a admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.048656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9174d-23ac-41d0-bdfa-872e3ab3314a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.048937 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9174d-23ac-41d0-bdfa-872e3ab3314a admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.049668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9174d-23ac-41d0-bdfa-872e3ab3314a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.050126 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-7fe5eebb-7c0b-4301-9be2-2effa25c9991 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:34.051589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7fe5eebb-7c0b-4301-9be2-2effa25c9991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:34.051712 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9174d-23ac-41d0-bdfa-872e3ab3314a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.060649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1370/2759] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:34.061840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5eebb-7c0b-4301-9be2-2effa25c9991 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:34.082150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5eebb-7c0b-4301-9be2-2effa25c9991 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.089904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ff9af-c2c5-437a-b944-39e3579190da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.089904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff9af-c2c5-437a-b944-39e3579190da admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.089904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff9af-c2c5-437a-b944-39e3579190da admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.089904 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff9af-c2c5-437a-b944-39e3579190da admin admin] PATH_INFO: `/v3/projects/98f70aa6464146fd9c128deea9bf1c5b/users/134e3b513fc94678999784f92c6823e9/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.106589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ff9af-c2c5-437a-b944-39e3579190da admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f70aa6464146fd9c128deea9bf1c5b, user_id=134e3b513fc94678999784f92c6823e9, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.117324 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ff9af-c2c5-437a-b944-39e3579190da admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.146252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1390/2760] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:34 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:34.147158 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9174d-23ac-41d0-bdfa-872e3ab3314a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:34.150753 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-12d9174d-23ac-41d0-bdfa-872e3ab3314a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:34.192162 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1371/2761] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:34 2022] PUT /identity/v3/projects/98f70aa6464146fd9c128deea9bf1c5b/users/134e3b513fc94678999784f92c6823e9/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:34.197687 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ff9af-c2c5-437a-b944-39e3579190da admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:34.213849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9174d-23ac-41d0-bdfa-872e3ab3314a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.265686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:34.266610 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb32c5f-72a4-44a7-be4b-d7bff84049a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.267584 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32c5f-72a4-44a7-be4b-d7bff84049a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.267864 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32c5f-72a4-44a7-be4b-d7bff84049a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.268136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32c5f-72a4-44a7-be4b-d7bff84049a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.270302 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ff9af-c2c5-437a-b944-39e3579190da admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.279036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258b126-a354-45c6-a034-7954d1affd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.279036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258b126-a354-45c6-a034-7954d1affd3d admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.279036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258b126-a354-45c6-a034-7954d1affd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.279036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258b126-a354-45c6-a034-7954d1affd3d admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.279437 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258b126-a354-45c6-a034-7954d1affd3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.279542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb32c5f-72a4-44a7-be4b-d7bff84049a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.279650 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258b126-a354-45c6-a034-7954d1affd3d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.279743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb32c5f-72a4-44a7-be4b-d7bff84049a0 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.288532 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1391/2762] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:34.289858 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fa4a8-49b8-4cf0-ae1a-bd0f3f333b58 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.290141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fa4a8-49b8-4cf0-ae1a-bd0f3f333b58 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.290413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fa4a8-49b8-4cf0-ae1a-bd0f3f333b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.317958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-585fa4a8-49b8-4cf0-ae1a-bd0f3f333b58 None None] MFA Rules not processed for user `134e3b513fc94678999784f92c6823e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:34.319092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-585fa4a8-49b8-4cf0-ae1a-bd0f3f333b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:34.387336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1392/2763] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:34.393264 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1372/2764] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:34 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:34.394965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fa4a8-49b8-4cf0-ae1a-bd0f3f333b58 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:34.399155 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258b126-a354-45c6-a034-7954d1affd3d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:34.400969 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6258b126-a354-45c6-a034-7954d1affd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:34.406660 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fa4a8-49b8-4cf0-ae1a-bd0f3f333b58 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.411519 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2713c-67db-492a-9d7b-6452c88f9f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.412191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2713c-67db-492a-9d7b-6452c88f9f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.412511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2713c-67db-492a-9d7b-6452c88f9f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.412811 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2713c-67db-492a-9d7b-6452c88f9f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.413579 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2713c-67db-492a-9d7b-6452c88f9f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.415204 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2713c-67db-492a-9d7b-6452c88f9f11 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.425958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1393/2765] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:34.429901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2713c-67db-492a-9d7b-6452c88f9f11 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:34.439713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2713c-67db-492a-9d7b-6452c88f9f11 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.445124 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227bcb3-9be8-40fd-be13-6890b15a8ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.445809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227bcb3-9be8-40fd-be13-6890b15a8ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.446092 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227bcb3-9be8-40fd-be13-6890b15a8ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.446463 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227bcb3-9be8-40fd-be13-6890b15a8ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.450011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227bcb3-9be8-40fd-be13-6890b15a8ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.450011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227bcb3-9be8-40fd-be13-6890b15a8ca9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.459141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1394/2766] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:52:34.463271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227bcb3-9be8-40fd-be13-6890b15a8ca9 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:34.473749 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227bcb3-9be8-40fd-be13-6890b15a8ca9 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.479243 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958d567-07e3-443c-95f2-23491c096b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.480684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958d567-07e3-443c-95f2-23491c096b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.480996 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958d567-07e3-443c-95f2-23491c096b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.481316 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958d567-07e3-443c-95f2-23491c096b13 admin admin] PATH_INFO: `/v3/projects/c8efb9b9a158481f9f9c521d765f48fb/users/af497df0400a40228badefeada1d6460/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.492661 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258b126-a354-45c6-a034-7954d1affd3d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.498201 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f2e29-6baa-4967-b057-684ea4f2a694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.498854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f2e29-6baa-4967-b057-684ea4f2a694 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.499039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958d567-07e3-443c-95f2-23491c096b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8efb9b9a158481f9f9c521d765f48fb, user_id=af497df0400a40228badefeada1d6460, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.499170 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f2e29-6baa-4967-b057-684ea4f2a694 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.499399 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f2e29-6baa-4967-b057-684ea4f2a694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.502178 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958d567-07e3-443c-95f2-23491c096b13 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.508825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f2e29-6baa-4967-b057-684ea4f2a694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.510319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f2e29-6baa-4967-b057-684ea4f2a694 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.520831 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1373/2767] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:34 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:52:34.619108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1395/2768] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:34 2022] PUT /identity/v3/projects/c8efb9b9a158481f9f9c521d765f48fb/users/af497df0400a40228badefeada1d6460/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:34.631347 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5481e-c149-4f6b-bd68-d95e93dda68a None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.631916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5481e-c149-4f6b-bd68-d95e93dda68a None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.632204 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5481e-c149-4f6b-bd68-d95e93dda68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.664070 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-53e5481e-c149-4f6b-bd68-d95e93dda68a None None] MFA Rules not processed for user `af497df0400a40228badefeada1d6460`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:34.665328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-53e5481e-c149-4f6b-bd68-d95e93dda68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:34.731735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1374/2769] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:34.736426 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958d567-07e3-443c-95f2-23491c096b13 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:34.800960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958d567-07e3-443c-95f2-23491c096b13 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.805900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a4c18-fcf6-4a37-a7bd-f1b790c774a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.806555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a4c18-fcf6-4a37-a7bd-f1b790c774a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.806836 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a4c18-fcf6-4a37-a7bd-f1b790c774a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.807108 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a4c18-fcf6-4a37-a7bd-f1b790c774a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.807806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a4c18-fcf6-4a37-a7bd-f1b790c774a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.809745 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a4c18-fcf6-4a37-a7bd-f1b790c774a0 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.851402 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5481e-c149-4f6b-bd68-d95e93dda68a None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:34.853911 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-53e5481e-c149-4f6b-bd68-d95e93dda68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:34.878365 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:34.885113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5481e-c149-4f6b-bd68-d95e93dda68a None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:34.890760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:34.891816 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:34.892103 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:34.892375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] PATH_INFO: `/v3/projects/6968eb6fafce4e01a4626c6aabf09c15` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:34.893643 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6968eb6fafce4e01a4626c6aabf09c15)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:34.906000 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:34.931755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1396/2770] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:34 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:34.932813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a4c18-fcf6-4a37-a7bd-f1b790c774a0 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:35.010698 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a4c18-fcf6-4a37-a7bd-f1b790c774a0 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:35.020349 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6968eb6fafce4e01a4626c6aabf09c15', 'request_id': 'req-885226cc-9ada-4865-8735-82f6ad6420fa'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:35.028812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6968eb6fafce4e01a4626c6aabf09c15', 'request_id': 'req-885226cc-9ada-4865-8735-82f6ad6420fa'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:35.035281 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6968eb6fafce4e01a4626c6aabf09c15', 'request_id': 'req-885226cc-9ada-4865-8735-82f6ad6420fa'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:35.078957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b5b6a-6aba-44df-90e0-529de7e6e689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:35.079633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b5b6a-6aba-44df-90e0-529de7e6e689 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:35.079910 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b5b6a-6aba-44df-90e0-529de7e6e689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:35.080179 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b5b6a-6aba-44df-90e0-529de7e6e689 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:35.080865 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b5b6a-6aba-44df-90e0-529de7e6e689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:35.082798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b5b6a-6aba-44df-90e0-529de7e6e689 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:35.122348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] The token cache is being invalidate because project 6968eb6fafce4e01a4626c6aabf09c15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:35.123279 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:35.123467 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:35.129203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1375/2771] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:34 2022] DELETE /identity/v3/projects/6968eb6fafce4e01a4626c6aabf09c15 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:35.191126 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1397/2772] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:34 2022] POST /identity/v3/users => generated 298 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:52:35.195303 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:35.197130 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:35.212523 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885226cc-9ada-4865-8735-82f6ad6420fa admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:35.227417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f0ecf-ca7f-432e-a51f-3057a5aaf378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:35.228274 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f0ecf-ca7f-432e-a51f-3057a5aaf378 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:35.228847 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f0ecf-ca7f-432e-a51f-3057a5aaf378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:35.229143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f0ecf-ca7f-432e-a51f-3057a5aaf378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:35.230088 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f0ecf-ca7f-432e-a51f-3057a5aaf378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:35.232068 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f0ecf-ca7f-432e-a51f-3057a5aaf378 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:35.244888 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1376/2773] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:35.251162 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b5b6a-6aba-44df-90e0-529de7e6e689 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:35.262578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b5b6a-6aba-44df-90e0-529de7e6e689 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:35.267781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb52fe-0ec0-4ea8-b379-a2eb736b82d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:35.268690 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb52fe-0ec0-4ea8-b379-a2eb736b82d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:35.268974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb52fe-0ec0-4ea8-b379-a2eb736b82d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:35.269287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb52fe-0ec0-4ea8-b379-a2eb736b82d9 admin admin] PATH_INFO: `/v3/projects/8d12c4267ab64e6f89e8cba3f1c74a3d/users/2dbbf999da60434d99cb7063f1035ed8/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:35.288240 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:35.296375 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb52fe-0ec0-4ea8-b379-a2eb736b82d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d12c4267ab64e6f89e8cba3f1c74a3d, user_id=2dbbf999da60434d99cb7063f1035ed8, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:35.299091 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb52fe-0ec0-4ea8-b379-a2eb736b82d9 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:35.414173 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1398/2774] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:35 2022] PUT /identity/v3/projects/8d12c4267ab64e6f89e8cba3f1c74a3d/users/2dbbf999da60434d99cb7063f1035ed8/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:35.418749 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f0ecf-ca7f-432e-a51f-3057a5aaf378 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:35.429022 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f0ecf-ca7f-432e-a51f-3057a5aaf378 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:35.486583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc369c18-2074-4497-9a2d-f00505d7bde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:35.490935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc369c18-2074-4497-9a2d-f00505d7bde8 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:35.490935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc369c18-2074-4497-9a2d-f00505d7bde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:35.490935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc369c18-2074-4497-9a2d-f00505d7bde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:35.490935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc369c18-2074-4497-9a2d-f00505d7bde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:35.490935 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc369c18-2074-4497-9a2d-f00505d7bde8 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:35.502376 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1377/2775] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:35.506562 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb52fe-0ec0-4ea8-b379-a2eb736b82d9 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:35.518171 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb52fe-0ec0-4ea8-b379-a2eb736b82d9 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:35.523009 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab06d44-89df-4fe8-a28b-0e5bde21aca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:35.523885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab06d44-89df-4fe8-a28b-0e5bde21aca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:35.524160 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab06d44-89df-4fe8-a28b-0e5bde21aca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:35.524427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab06d44-89df-4fe8-a28b-0e5bde21aca2 admin admin] PATH_INFO: `/v3/projects/8d12c4267ab64e6f89e8cba3f1c74a3d/users/2dbbf999da60434d99cb7063f1035ed8/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:35.526379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab06d44-89df-4fe8-a28b-0e5bde21aca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d12c4267ab64e6f89e8cba3f1c74a3d, user_id=2dbbf999da60434d99cb7063f1035ed8, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:35.528957 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab06d44-89df-4fe8-a28b-0e5bde21aca2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:35.539060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1399/2776] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:35 2022] PUT /identity/v3/projects/8d12c4267ab64e6f89e8cba3f1c74a3d/users/2dbbf999da60434d99cb7063f1035ed8/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:35.550559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba452f9-4fcc-496b-83b4-d268004df88f None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:35.550852 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba452f9-4fcc-496b-83b4-d268004df88f None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:35.551137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba452f9-4fcc-496b-83b4-d268004df88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:35.578803 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-cba452f9-4fcc-496b-83b4-d268004df88f None None] MFA Rules not processed for user `2dbbf999da60434d99cb7063f1035ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:35.580041 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cba452f9-4fcc-496b-83b4-d268004df88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:35.590792 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:35.653899 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1378/2777] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:35.664228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:35.665626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab06d44-89df-4fe8-a28b-0e5bde21aca2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:35.667504 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-aab06d44-89df-4fe8-a28b-0e5bde21aca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:35.725591 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab06d44-89df-4fe8-a28b-0e5bde21aca2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:35.729305 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edc393-b304-491a-bb32-a8da427912e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:35.729945 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc393-b304-491a-bb32-a8da427912e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:35.730227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc393-b304-491a-bb32-a8da427912e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:35.730499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc393-b304-491a-bb32-a8da427912e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:35.740278 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edc393-b304-491a-bb32-a8da427912e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:35.741868 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edc393-b304-491a-bb32-a8da427912e7 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:35.752779 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1400/2778] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:52:35.912023 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba452f9-4fcc-496b-83b4-d268004df88f None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:35.922291 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba452f9-4fcc-496b-83b4-d268004df88f None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:35.925845 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6967ec-33c9-48e6-a9c8-47a2ee79dbda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:35.926456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6967ec-33c9-48e6-a9c8-47a2ee79dbda service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:35.926735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6967ec-33c9-48e6-a9c8-47a2ee79dbda service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:35.927007 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6967ec-33c9-48e6-a9c8-47a2ee79dbda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:35.928744 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-4f6967ec-33c9-48e6-a9c8-47a2ee79dbda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:35.938937 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:35.986843 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6967ec-33c9-48e6-a9c8-47a2ee79dbda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:35.988380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6967ec-33c9-48e6-a9c8-47a2ee79dbda service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:35.997466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1379/2779] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:36.508878 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edc393-b304-491a-bb32-a8da427912e7 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:36.576228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edc393-b304-491a-bb32-a8da427912e7 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:36.581343 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:36.582331 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:36.582522 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:36.582802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] PATH_INFO: `/v3/users/af497df0400a40228badefeada1d6460` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:36.584151 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] RBAC: Authorizing `identity:delete_user(user_id=af497df0400a40228badefeada1d6460)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:36.586259 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:36.763119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af497df0400a40228badefeada1d6460'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:36.770120 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af497df0400a40228badefeada1d6460'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:36.781599 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af497df0400a40228badefeada1d6460'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:36.852242 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af497df0400a40228badefeada1d6460'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:36.852983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af497df0400a40228badefeada1d6460'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:36.859569 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1401/2780] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:36 2022] DELETE /identity/v3/users/af497df0400a40228badefeada1d6460 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:36.866889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c39e64-5ec9-45ac-b34f-b26d37d9b5de None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:36.866889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c39e64-5ec9-45ac-b34f-b26d37d9b5de None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:36.866889 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c39e64-5ec9-45ac-b34f-b26d37d9b5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:36.883247 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-81c39e64-5ec9-45ac-b34f-b26d37d9b5de None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:36.884556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-81c39e64-5ec9-45ac-b34f-b26d37d9b5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:36.943462 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1380/2781] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:36.953581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:36.954890 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:36.960236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:37.012447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a5154-93e9-4d01-b6aa-6c5b7f15177a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:37.016469 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f760f14-72be-49db-8291-5285887b1716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:37.017123 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f760f14-72be-49db-8291-5285887b1716 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:37.017439 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f760f14-72be-49db-8291-5285887b1716 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:37.017718 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f760f14-72be-49db-8291-5285887b1716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:37.027470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f760f14-72be-49db-8291-5285887b1716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:37.028979 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f760f14-72be-49db-8291-5285887b1716 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:37.039084 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1402/2782] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:37.497095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c39e64-5ec9-45ac-b34f-b26d37d9b5de None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:37.497095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-81c39e64-5ec9-45ac-b34f-b26d37d9b5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:37.508089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c39e64-5ec9-45ac-b34f-b26d37d9b5de None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:37.513586 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:37.514534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:37.514820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:37.515095 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] PATH_INFO: `/v3/projects/c8efb9b9a158481f9f9c521d765f48fb` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:37.516353 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8efb9b9a158481f9f9c521d765f48fb)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:37.518294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:37.631067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8efb9b9a158481f9f9c521d765f48fb', 'request_id': 'req-ea54ff7b-a385-42bd-9326-2e5093aa3d16'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:37.637514 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8efb9b9a158481f9f9c521d765f48fb', 'request_id': 'req-ea54ff7b-a385-42bd-9326-2e5093aa3d16'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:37.648198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8efb9b9a158481f9f9c521d765f48fb', 'request_id': 'req-ea54ff7b-a385-42bd-9326-2e5093aa3d16'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:37.726540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] The token cache is being invalidate because project c8efb9b9a158481f9f9c521d765f48fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:37.726927 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:37.727589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:37.729174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1381/2783] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:37 2022] DELETE /identity/v3/projects/c8efb9b9a158481f9f9c521d765f48fb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:37.734040 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f760f14-72be-49db-8291-5285887b1716 service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:37.743793 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f760f14-72be-49db-8291-5285887b1716 service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:37.799748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:37.800625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:37.800905 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:37.801202 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] PATH_INFO: `/v3/users/2dbbf999da60434d99cb7063f1035ed8` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:37.802413 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dbbf999da60434d99cb7063f1035ed8)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:37.804329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:37.933699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dbbf999da60434d99cb7063f1035ed8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:37.940743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dbbf999da60434d99cb7063f1035ed8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:37.953039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dbbf999da60434d99cb7063f1035ed8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:37.994899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dbbf999da60434d99cb7063f1035ed8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:37.995460 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-692f74bc-96e1-4f98-85b2-bfd0a2183eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dbbf999da60434d99cb7063f1035ed8'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:38.002558 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1403/2784] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:37 2022] DELETE /identity/v3/users/2dbbf999da60434d99cb7063f1035ed8 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:38.454540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:38.455144 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:38.472644 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54ff7b-a385-42bd-9326-2e5093aa3d16 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:38.508784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:38.534412 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:38.535325 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:38.535555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:38.535834 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] PATH_INFO: `/v3/projects/8d12c4267ab64e6f89e8cba3f1c74a3d` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:38.537116 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d12c4267ab64e6f89e8cba3f1c74a3d)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:38.539067 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:38.640648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d12c4267ab64e6f89e8cba3f1c74a3d', 'request_id': 'req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:38.647420 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d12c4267ab64e6f89e8cba3f1c74a3d', 'request_id': 'req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:38.654060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d12c4267ab64e6f89e8cba3f1c74a3d', 'request_id': 'req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:38.714674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] The token cache is being invalidate because project 8d12c4267ab64e6f89e8cba3f1c74a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:38.714970 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:38.716194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-45aff9e7-d1e2-4cf9-aa79-2f85e2013991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:38.717825 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1382/2785] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:38 2022] DELETE /identity/v3/projects/8d12c4267ab64e6f89e8cba3f1c74a3d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:38.726392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644e45f-2aeb-4ec8-aa73-a1bd23899a42 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:38.726684 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644e45f-2aeb-4ec8-aa73-a1bd23899a42 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:38.726960 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644e45f-2aeb-4ec8-aa73-a1bd23899a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:38.744975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-6644e45f-2aeb-4ec8-aa73-a1bd23899a42 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:38.746221 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-6644e45f-2aeb-4ec8-aa73-a1bd23899a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:38.766204 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:38.821029 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1404/2786] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:38.826535 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff3da7-924c-45f0-81ba-972f08c22e35 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:38.826870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff3da7-924c-45f0-81ba-972f08c22e35 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:38.827143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff3da7-924c-45f0-81ba-972f08c22e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:38.846240 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-51ff3da7-924c-45f0-81ba-972f08c22e35 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:38.847434 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-51ff3da7-924c-45f0-81ba-972f08c22e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:38.855441 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1383/2787] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:38.859842 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644e45f-2aeb-4ec8-aa73-a1bd23899a42 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:38.870595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644e45f-2aeb-4ec8-aa73-a1bd23899a42 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:38.875925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae015dc9-1c34-4c84-950f-1bbcffce18af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:38.876585 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae015dc9-1c34-4c84-950f-1bbcffce18af admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:38.876862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae015dc9-1c34-4c84-950f-1bbcffce18af admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:38.877133 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae015dc9-1c34-4c84-950f-1bbcffce18af admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:38.877913 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae015dc9-1c34-4c84-950f-1bbcffce18af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:52:38.878191 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae015dc9-1c34-4c84-950f-1bbcffce18af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:38.879763 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae015dc9-1c34-4c84-950f-1bbcffce18af admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:38.893772 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1405/2788] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:52:38.898556 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff3da7-924c-45f0-81ba-972f08c22e35 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:38.908308 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff3da7-924c-45f0-81ba-972f08c22e35 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:38.913152 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e45df-2eb1-4da2-b426-72fa9319a9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:38.913877 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e45df-2eb1-4da2-b426-72fa9319a9ce admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:38.914159 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e45df-2eb1-4da2-b426-72fa9319a9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:38.914441 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e45df-2eb1-4da2-b426-72fa9319a9ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:38.915138 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e45df-2eb1-4da2-b426-72fa9319a9ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:38.917076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e45df-2eb1-4da2-b426-72fa9319a9ce admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:39.003436 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1384/2789] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:38 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:39.007417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae015dc9-1c34-4c84-950f-1bbcffce18af admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:39.018327 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae015dc9-1c34-4c84-950f-1bbcffce18af admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:39.072685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854808b-6766-41de-acc5-cffb66790c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:39.073296 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854808b-6766-41de-acc5-cffb66790c38 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:39.073578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854808b-6766-41de-acc5-cffb66790c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:39.073852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854808b-6766-41de-acc5-cffb66790c38 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:39.074550 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854808b-6766-41de-acc5-cffb66790c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:39.077103 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854808b-6766-41de-acc5-cffb66790c38 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:39.138254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1406/2790] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:39 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:39.142695 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e45df-2eb1-4da2-b426-72fa9319a9ce admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:39.153862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e45df-2eb1-4da2-b426-72fa9319a9ce admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:39.158732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24babf03-23a8-48ae-ba1a-979e2071102d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:39.159421 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24babf03-23a8-48ae-ba1a-979e2071102d admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:39.159707 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24babf03-23a8-48ae-ba1a-979e2071102d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:39.159986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24babf03-23a8-48ae-ba1a-979e2071102d admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:39.160716 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24babf03-23a8-48ae-ba1a-979e2071102d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:39.162358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24babf03-23a8-48ae-ba1a-979e2071102d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:39.172981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1385/2791] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:39.177002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854808b-6766-41de-acc5-cffb66790c38 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:39.187726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854808b-6766-41de-acc5-cffb66790c38 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:39.192364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977533aa-4ccf-491d-b543-64d06fe03a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:39.193012 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977533aa-4ccf-491d-b543-64d06fe03a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:39.193333 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977533aa-4ccf-491d-b543-64d06fe03a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:39.196857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977533aa-4ccf-491d-b543-64d06fe03a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:39.196857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977533aa-4ccf-491d-b543-64d06fe03a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:39.196857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977533aa-4ccf-491d-b543-64d06fe03a4b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:39.206856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1407/2792] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:39.211125 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24babf03-23a8-48ae-ba1a-979e2071102d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:39.221201 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24babf03-23a8-48ae-ba1a-979e2071102d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:39.225916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170310e-101e-4c2c-a0cf-370fe1a0a2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:39.226781 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170310e-101e-4c2c-a0cf-370fe1a0a2c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:39.227063 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170310e-101e-4c2c-a0cf-370fe1a0a2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:39.227335 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170310e-101e-4c2c-a0cf-370fe1a0a2c8 admin admin] PATH_INFO: `/v3/projects/43570714c9364a02ad1a807c9d883954/users/bf44ca529fb34d068d9f89e1748e8c51/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:39.243071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170310e-101e-4c2c-a0cf-370fe1a0a2c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43570714c9364a02ad1a807c9d883954, user_id=bf44ca529fb34d068d9f89e1748e8c51, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:39.245821 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170310e-101e-4c2c-a0cf-370fe1a0a2c8 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:39.292184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1386/2793] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:39 2022] PUT /identity/v3/projects/43570714c9364a02ad1a807c9d883954/users/bf44ca529fb34d068d9f89e1748e8c51/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:39.303968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080098e-7fb4-49b6-9013-a2a18f9f4dfa None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:39.303968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080098e-7fb4-49b6-9013-a2a18f9f4dfa None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:39.304213 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080098e-7fb4-49b6-9013-a2a18f9f4dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:39.332512 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-5080098e-7fb4-49b6-9013-a2a18f9f4dfa None None] MFA Rules not processed for user `bf44ca529fb34d068d9f89e1748e8c51`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:39.333695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5080098e-7fb4-49b6-9013-a2a18f9f4dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:39.400003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1408/2794] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:39.411036 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170310e-101e-4c2c-a0cf-370fe1a0a2c8 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:39.412864 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-1170310e-101e-4c2c-a0cf-370fe1a0a2c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:39.476043 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170310e-101e-4c2c-a0cf-370fe1a0a2c8 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:39.480498 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb939f-d952-4d1c-b3e3-85fe0bb66d7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:39.481647 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb939f-d952-4d1c-b3e3-85fe0bb66d7e service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:39.481934 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb939f-d952-4d1c-b3e3-85fe0bb66d7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:39.482209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb939f-d952-4d1c-b3e3-85fe0bb66d7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:39.497469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb939f-d952-4d1c-b3e3-85fe0bb66d7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:39.498924 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb939f-d952-4d1c-b3e3-85fe0bb66d7e service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:39.509418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1387/2795] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:43.417834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080098e-7fb4-49b6-9013-a2a18f9f4dfa None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:43.419395 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-5080098e-7fb4-49b6-9013-a2a18f9f4dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:43.452111 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:43.495816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080098e-7fb4-49b6-9013-a2a18f9f4dfa None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:43.500732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:43.501742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:43.502018 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:43.502336 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] PATH_INFO: `/v3/users/19bef42d11c14fb2834908330a8ab4a5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:43.503686 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19bef42d11c14fb2834908330a8ab4a5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:43.506369 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:43.582862 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb939f-d952-4d1c-b3e3-85fe0bb66d7e service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:43.589704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:43.594809 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb939f-d952-4d1c-b3e3-85fe0bb66d7e service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:43.599481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:43.603499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:43.603499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:43.603499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] PATH_INFO: `/v3/users/bf44ca529fb34d068d9f89e1748e8c51` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:43.603499 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf44ca529fb34d068d9f89e1748e8c51)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:43.604162 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:43.729743 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19bef42d11c14fb2834908330a8ab4a5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:43.737590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19bef42d11c14fb2834908330a8ab4a5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:43.749834 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19bef42d11c14fb2834908330a8ab4a5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:43.840035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19bef42d11c14fb2834908330a8ab4a5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:43.840728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19bef42d11c14fb2834908330a8ab4a5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:43.847482 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1409/2796] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:43 2022] DELETE /identity/v3/users/19bef42d11c14fb2834908330a8ab4a5 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:43.849325 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:43.849869 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:43.856451 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf44ca529fb34d068d9f89e1748e8c51'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:43.857203 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf44ca529fb34d068d9f89e1748e8c51'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:43.871603 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf44ca529fb34d068d9f89e1748e8c51'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:43.880727 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf44ca529fb34d068d9f89e1748e8c51'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:43.919556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991b4fb-9087-4489-842f-0f84c5f3b512 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:43.923816 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2fc73-f67b-4a55-88cf-8a9f7694c0e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:43.924477 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2fc73-f67b-4a55-88cf-8a9f7694c0e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:43.924794 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2fc73-f67b-4a55-88cf-8a9f7694c0e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:43.925070 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2fc73-f67b-4a55-88cf-8a9f7694c0e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:43.926923 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3fd2fc73-f67b-4a55-88cf-8a9f7694c0e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:43.978802 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:43.983319 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5290f2a4-fe6a-43f0-b498-a3eede858c7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf44ca529fb34d068d9f89e1748e8c51'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:43.991171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1388/2797] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:43 2022] DELETE /identity/v3/users/bf44ca529fb34d068d9f89e1748e8c51 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:43.992671 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49d250-2933-42be-a7e3-b5315aeff641 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:43.992968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49d250-2933-42be-a7e3-b5315aeff641 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:43.993292 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49d250-2933-42be-a7e3-b5315aeff641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:43.998232 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2fc73-f67b-4a55-88cf-8a9f7694c0e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:43.999720 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2fc73-f67b-4a55-88cf-8a9f7694c0e8 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:44.010195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1410/2798] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:44.011495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68449a98-0b47-44dd-a11b-4771de22e1bf None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:44.011769 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-6a49d250-2933-42be-a7e3-b5315aeff641 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:44.011928 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68449a98-0b47-44dd-a11b-4771de22e1bf None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:44.012039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68449a98-0b47-44dd-a11b-4771de22e1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:44.012997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6a49d250-2933-42be-a7e3-b5315aeff641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:44.030096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-68449a98-0b47-44dd-a11b-4771de22e1bf None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:44.032169 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-68449a98-0b47-44dd-a11b-4771de22e1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:44.079840 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1389/2799] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:44.090414 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49d250-2933-42be-a7e3-b5315aeff641 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:44.092025 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-6a49d250-2933-42be-a7e3-b5315aeff641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:44.108001 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1411/2800] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:44.123110 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68449a98-0b47-44dd-a11b-4771de22e1bf None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:44.124706 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-68449a98-0b47-44dd-a11b-4771de22e1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:44.159143 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49d250-2933-42be-a7e3-b5315aeff641 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:44.163506 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cac43-e731-4d6f-aaf6-2656b308791f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:44.164202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cac43-e731-4d6f-aaf6-2656b308791f service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:44.165151 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cac43-e731-4d6f-aaf6-2656b308791f service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:44.165479 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cac43-e731-4d6f-aaf6-2656b308791f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:44.177881 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867cac43-e731-4d6f-aaf6-2656b308791f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:44.181299 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867cac43-e731-4d6f-aaf6-2656b308791f service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:44.185748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68449a98-0b47-44dd-a11b-4771de22e1bf None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:44.189964 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1390/2801] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:44.193949 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e1e24-123f-4cf2-bbb9-df405a31395e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:44.194650 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e1e24-123f-4cf2-bbb9-df405a31395e service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:44.194934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e1e24-123f-4cf2-bbb9-df405a31395e service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:44.195208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e1e24-123f-4cf2-bbb9-df405a31395e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:44.205813 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e1e24-123f-4cf2-bbb9-df405a31395e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:44.207252 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e1e24-123f-4cf2-bbb9-df405a31395e service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:44.217496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1412/2802] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:52:44.656795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cac43-e731-4d6f-aaf6-2656b308791f service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:44.657270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-867cac43-e731-4d6f-aaf6-2656b308791f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:44.670243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cac43-e731-4d6f-aaf6-2656b308791f service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:44.675431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:44.676377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:44.676853 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:44.677134 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] PATH_INFO: `/v3/projects/5c08a4ef4b4d4e368a5df4a11ca3b89e` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:44.678389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c08a4ef4b4d4e368a5df4a11ca3b89e)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:44.680388 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:44.700316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:44.802390 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e1e24-123f-4cf2-bbb9-df405a31395e service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:44.804450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3c7e1e24-123f-4cf2-bbb9-df405a31395e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:44.845002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e1e24-123f-4cf2-bbb9-df405a31395e service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:44.848836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c08a4ef4b4d4e368a5df4a11ca3b89e', 'request_id': 'req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:44.858674 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c08a4ef4b4d4e368a5df4a11ca3b89e', 'request_id': 'req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:44.875017 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c08a4ef4b4d4e368a5df4a11ca3b89e', 'request_id': 'req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:44.931054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:44.932054 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:44.932402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:44.932737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] PATH_INFO: `/v3/projects/43570714c9364a02ad1a807c9d883954` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:44.934229 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43570714c9364a02ad1a807c9d883954)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:44.936323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:45.012166 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] The token cache is being invalidate because project 5c08a4ef4b4d4e368a5df4a11ca3b89e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:45.019797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.019797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.019797 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1391/2803] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:44 2022] DELETE /identity/v3/projects/5c08a4ef4b4d4e368a5df4a11ca3b89e => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:45.021355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:45.023806 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:45.038836 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ffe4b-d3e4-4e87-a3e6-4a602000e86d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:45.045925 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:45.058496 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43570714c9364a02ad1a807c9d883954', 'request_id': 'req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.100655 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:45.101684 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:45.102100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:45.102443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] PATH_INFO: `/v3/users/6429f92b28894cc695a65a16eead4c3c` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:45.103752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6429f92b28894cc695a65a16eead4c3c)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:45.105804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:45.143567 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43570714c9364a02ad1a807c9d883954', 'request_id': 'req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.150547 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43570714c9364a02ad1a807c9d883954', 'request_id': 'req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.165671 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] The token cache is being invalidate because project 43570714c9364a02ad1a807c9d883954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:45.166189 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.167057 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.168855 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1413/2804] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:44 2022] DELETE /identity/v3/projects/43570714c9364a02ad1a807c9d883954 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:45.322760 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6429f92b28894cc695a65a16eead4c3c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.323732 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6429f92b28894cc695a65a16eead4c3c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.342350 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6429f92b28894cc695a65a16eead4c3c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.350759 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6429f92b28894cc695a65a16eead4c3c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.424018 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-6282a14f-1273-4f3c-a522-74808ce6cd9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6429f92b28894cc695a65a16eead4c3c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:45.434378 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1392/2805] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:45 2022] DELETE /identity/v3/users/6429f92b28894cc695a65a16eead4c3c => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:45.935214 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:45.937271 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:45.954783 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62caaf-2ebd-43fb-a6e3-2b7cd2a18873 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:46.033381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:46.034412 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:46.034695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:46.035112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] PATH_INFO: `/v3/projects/809fa42e8a964fc88f644c590ac6e855` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:46.036334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=809fa42e8a964fc88f644c590ac6e855)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:46.038378 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:46.163231 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '809fa42e8a964fc88f644c590ac6e855', 'request_id': 'req-9b8160b4-f705-4db3-be04-463d0f4d4a93'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:46.246462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '809fa42e8a964fc88f644c590ac6e855', 'request_id': 'req-9b8160b4-f705-4db3-be04-463d0f4d4a93'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:46.252736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '809fa42e8a964fc88f644c590ac6e855', 'request_id': 'req-9b8160b4-f705-4db3-be04-463d0f4d4a93'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:46.264364 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] The token cache is being invalidate because project 809fa42e8a964fc88f644c590ac6e855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:52:46.264651 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:46.268435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-9b8160b4-f705-4db3-be04-463d0f4d4a93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:46.268435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1414/2806] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:52:45 2022] DELETE /identity/v3/projects/809fa42e8a964fc88f644c590ac6e855 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:46.272558 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ead1e-2866-4a1e-93a9-c914a2eb7577 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:46.273099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ead1e-2866-4a1e-93a9-c914a2eb7577 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:46.273242 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ead1e-2866-4a1e-93a9-c914a2eb7577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:46.292681 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-7b8ead1e-2866-4a1e-93a9-c914a2eb7577 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:46.293932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7b8ead1e-2866-4a1e-93a9-c914a2eb7577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:46.311015 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:46.356943 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1393/2807] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:46.364971 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece56f3-df1a-487b-95ec-c798e51838c1 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:46.365290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece56f3-df1a-487b-95ec-c798e51838c1 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:46.365570 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece56f3-df1a-487b-95ec-c798e51838c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:46.386039 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-eece56f3-df1a-487b-95ec-c798e51838c1 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:46.387250 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-eece56f3-df1a-487b-95ec-c798e51838c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:46.396666 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1415/2808] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:46.400854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ead1e-2866-4a1e-93a9-c914a2eb7577 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:46.411254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ead1e-2866-4a1e-93a9-c914a2eb7577 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:46.416003 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac73e35-1cf1-4d43-af5d-8c2b06805fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:46.416669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac73e35-1cf1-4d43-af5d-8c2b06805fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:46.416981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac73e35-1cf1-4d43-af5d-8c2b06805fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:46.417287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac73e35-1cf1-4d43-af5d-8c2b06805fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:46.418027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac73e35-1cf1-4d43-af5d-8c2b06805fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:52:46.418329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac73e35-1cf1-4d43-af5d-8c2b06805fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:46.419968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac73e35-1cf1-4d43-af5d-8c2b06805fa8 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:46.438817 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece56f3-df1a-487b-95ec-c798e51838c1 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:46.441536 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1394/2809] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:52:46.449982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece56f3-df1a-487b-95ec-c798e51838c1 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:46.455186 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b46a70-4659-4f4c-83b7-6575ac35aeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:46.455848 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b46a70-4659-4f4c-83b7-6575ac35aeec admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:46.456128 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b46a70-4659-4f4c-83b7-6575ac35aeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:46.456448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b46a70-4659-4f4c-83b7-6575ac35aeec admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:46.457148 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b46a70-4659-4f4c-83b7-6575ac35aeec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:46.459064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b46a70-4659-4f4c-83b7-6575ac35aeec admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:46.544143 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1416/2810] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:46 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:52:46.549696 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac73e35-1cf1-4d43-af5d-8c2b06805fa8 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:46.560992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac73e35-1cf1-4d43-af5d-8c2b06805fa8 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:46.636668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d174ce9-5e8f-4908-a439-fdcb363f0afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:46.637390 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d174ce9-5e8f-4908-a439-fdcb363f0afb admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:46.637677 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d174ce9-5e8f-4908-a439-fdcb363f0afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:46.637948 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d174ce9-5e8f-4908-a439-fdcb363f0afb admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:46.638668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d174ce9-5e8f-4908-a439-fdcb363f0afb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:46.641262 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d174ce9-5e8f-4908-a439-fdcb363f0afb admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:46.762137 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1395/2811] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:46 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:46.765493 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b46a70-4659-4f4c-83b7-6575ac35aeec admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:46.776188 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b46a70-4659-4f4c-83b7-6575ac35aeec admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:46.780669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeba0d5-bf88-4601-b45e-8666a24c306d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:46.781356 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeba0d5-bf88-4601-b45e-8666a24c306d admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:46.781636 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeba0d5-bf88-4601-b45e-8666a24c306d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:46.781938 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeba0d5-bf88-4601-b45e-8666a24c306d admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:46.782659 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeba0d5-bf88-4601-b45e-8666a24c306d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:46.784223 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeba0d5-bf88-4601-b45e-8666a24c306d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:46.794615 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1417/2812] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:46.799380 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d174ce9-5e8f-4908-a439-fdcb363f0afb admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:46.809812 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d174ce9-5e8f-4908-a439-fdcb363f0afb admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:46.815199 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3e053-5694-4910-978e-3f7af2a56f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:46.815961 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3e053-5694-4910-978e-3f7af2a56f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:46.816251 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3e053-5694-4910-978e-3f7af2a56f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:46.816529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3e053-5694-4910-978e-3f7af2a56f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:46.817296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3e053-5694-4910-978e-3f7af2a56f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:46.819944 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3e053-5694-4910-978e-3f7af2a56f95 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:46.835543 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1396/2813] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:52:46.836301 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeba0d5-bf88-4601-b45e-8666a24c306d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:46.846768 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeba0d5-bf88-4601-b45e-8666a24c306d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:46.851329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99695f5c-003b-43a2-8f69-d7d21c55ca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:46.852209 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99695f5c-003b-43a2-8f69-d7d21c55ca20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:46.852489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99695f5c-003b-43a2-8f69-d7d21c55ca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:46.852762 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99695f5c-003b-43a2-8f69-d7d21c55ca20 admin admin] PATH_INFO: `/v3/projects/65f66da0127c4b08ac88f7eaabacf184/users/211b9712181149088673e9ec04e82dc0/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:46.864675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:46.868862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99695f5c-003b-43a2-8f69-d7d21c55ca20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f66da0127c4b08ac88f7eaabacf184, user_id=211b9712181149088673e9ec04e82dc0, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:46.871575 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99695f5c-003b-43a2-8f69-d7d21c55ca20 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:46.966879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1418/2814] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:46 2022] PUT /identity/v3/projects/65f66da0127c4b08ac88f7eaabacf184/users/211b9712181149088673e9ec04e82dc0/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:46.978016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e21bf6-af58-4cb5-bb32-34b3eea4a88e None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:46.978458 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e21bf6-af58-4cb5-bb32-34b3eea4a88e None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:46.978893 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e21bf6-af58-4cb5-bb32-34b3eea4a88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:47.011213 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-45e21bf6-af58-4cb5-bb32-34b3eea4a88e None None] MFA Rules not processed for user `211b9712181149088673e9ec04e82dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:47.012550 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-45e21bf6-af58-4cb5-bb32-34b3eea4a88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:47.080950 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1397/2815] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:47.085060 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99695f5c-003b-43a2-8f69-d7d21c55ca20 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:47.149633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99695f5c-003b-43a2-8f69-d7d21c55ca20 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:47.154976 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa54f3-bcc4-4c4d-9825-2185216eefe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:47.156171 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa54f3-bcc4-4c4d-9825-2185216eefe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:47.156457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa54f3-bcc4-4c4d-9825-2185216eefe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:47.156754 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa54f3-bcc4-4c4d-9825-2185216eefe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:47.157471 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa54f3-bcc4-4c4d-9825-2185216eefe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:47.159374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa54f3-bcc4-4c4d-9825-2185216eefe6 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:47.245492 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1419/2816] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:52:47 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:47.248046 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e21bf6-af58-4cb5-bb32-34b3eea4a88e None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:47.261427 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e21bf6-af58-4cb5-bb32-34b3eea4a88e None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:47.275424 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:47.327224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58050f-9690-4e2f-9518-82bec59690a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:47.327986 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58050f-9690-4e2f-9518-82bec59690a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:47.328465 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58050f-9690-4e2f-9518-82bec59690a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:47.328788 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58050f-9690-4e2f-9518-82bec59690a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:47.329595 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58050f-9690-4e2f-9518-82bec59690a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:47.334464 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58050f-9690-4e2f-9518-82bec59690a4 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:47.464133 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1398/2817] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:52:47 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:52:47.468845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa54f3-bcc4-4c4d-9825-2185216eefe6 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:47.479897 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa54f3-bcc4-4c4d-9825-2185216eefe6 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:47.484874 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37aedf6-3203-438f-a214-499c746dbf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:47.491003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37aedf6-3203-438f-a214-499c746dbf7c admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:47.491003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37aedf6-3203-438f-a214-499c746dbf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:47.491003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37aedf6-3203-438f-a214-499c746dbf7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:47.491003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37aedf6-3203-438f-a214-499c746dbf7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:47.491003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37aedf6-3203-438f-a214-499c746dbf7c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:47.500595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1420/2818] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:52:47.505359 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58050f-9690-4e2f-9518-82bec59690a4 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:47.516851 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58050f-9690-4e2f-9518-82bec59690a4 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:47.522569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca550a2-198a-4372-8e5f-e70b6f6f4974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:47.523423 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca550a2-198a-4372-8e5f-e70b6f6f4974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:47.523706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca550a2-198a-4372-8e5f-e70b6f6f4974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:47.523983 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca550a2-198a-4372-8e5f-e70b6f6f4974 admin admin] PATH_INFO: `/v3/projects/acbe617ee75546c4a430c206dae14aae/users/23834a46141e4dbab69661ffbfbc6fc6/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:47.541563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca550a2-198a-4372-8e5f-e70b6f6f4974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acbe617ee75546c4a430c206dae14aae, user_id=23834a46141e4dbab69661ffbfbc6fc6, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:47.544614 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca550a2-198a-4372-8e5f-e70b6f6f4974 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:47.646362 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1399/2819] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:47 2022] PUT /identity/v3/projects/acbe617ee75546c4a430c206dae14aae/users/23834a46141e4dbab69661ffbfbc6fc6/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:52:47.647056 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37aedf6-3203-438f-a214-499c746dbf7c admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:47.658748 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37aedf6-3203-438f-a214-499c746dbf7c admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:47.725974 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b2ee3-0439-4bb1-8f90-2d286a3622df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:47.726681 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2ee3-0439-4bb1-8f90-2d286a3622df admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:47.726975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2ee3-0439-4bb1-8f90-2d286a3622df admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:47.727254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2ee3-0439-4bb1-8f90-2d286a3622df admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:47.727987 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b2ee3-0439-4bb1-8f90-2d286a3622df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:47.729611 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b2ee3-0439-4bb1-8f90-2d286a3622df admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:47.739759 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1421/2820] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:52:47.744001 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca550a2-198a-4372-8e5f-e70b6f6f4974 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:47.754393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca550a2-198a-4372-8e5f-e70b6f6f4974 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:47.758997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c588c8-e5ce-471c-993f-f500b423ebd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:47.759954 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c588c8-e5ce-471c-993f-f500b423ebd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:47.760182 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c588c8-e5ce-471c-993f-f500b423ebd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:47.760392 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c588c8-e5ce-471c-993f-f500b423ebd3 admin admin] PATH_INFO: `/v3/projects/acbe617ee75546c4a430c206dae14aae/users/23834a46141e4dbab69661ffbfbc6fc6/roles/641b0deca0e94bb6bd15d6e839352262` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:47.762428 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c588c8-e5ce-471c-993f-f500b423ebd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acbe617ee75546c4a430c206dae14aae, user_id=23834a46141e4dbab69661ffbfbc6fc6, role_id=641b0deca0e94bb6bd15d6e839352262)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:47.766946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c588c8-e5ce-471c-993f-f500b423ebd3 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:47.785472 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1400/2821] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:52:47 2022] PUT /identity/v3/projects/acbe617ee75546c4a430c206dae14aae/users/23834a46141e4dbab69661ffbfbc6fc6/roles/641b0deca0e94bb6bd15d6e839352262 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:47.796094 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960c701-967b-44fa-8bb2-e6ecf57f2a6a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:47.796382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960c701-967b-44fa-8bb2-e6ecf57f2a6a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:47.796669 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960c701-967b-44fa-8bb2-e6ecf57f2a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:47.824374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-7960c701-967b-44fa-8bb2-e6ecf57f2a6a None None] MFA Rules not processed for user `23834a46141e4dbab69661ffbfbc6fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:47.825614 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-7960c701-967b-44fa-8bb2-e6ecf57f2a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:47.872020 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:47.894254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1422/2822] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:47.905987 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c588c8-e5ce-471c-993f-f500b423ebd3 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:47.907651 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-66c588c8-e5ce-471c-993f-f500b423ebd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:47.963607 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c588c8-e5ce-471c-993f-f500b423ebd3 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:47.967228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070357e0-2642-4d20-9b4c-7761a1c46d49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:47.967870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070357e0-2642-4d20-9b4c-7761a1c46d49 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:47.968176 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070357e0-2642-4d20-9b4c-7761a1c46d49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:47.968449 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070357e0-2642-4d20-9b4c-7761a1c46d49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:47.977413 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070357e0-2642-4d20-9b4c-7761a1c46d49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:47.978842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070357e0-2642-4d20-9b4c-7761a1c46d49 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:47.988167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1401/2823] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:48.179692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960c701-967b-44fa-8bb2-e6ecf57f2a6a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:48.189929 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960c701-967b-44fa-8bb2-e6ecf57f2a6a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:48.193442 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d4716-f6e8-4136-9a85-1279cb3ff4a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:48.194065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d4716-f6e8-4136-9a85-1279cb3ff4a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:48.194347 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d4716-f6e8-4136-9a85-1279cb3ff4a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:48.194620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d4716-f6e8-4136-9a85-1279cb3ff4a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:48.252556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d4716-f6e8-4136-9a85-1279cb3ff4a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:48.254112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d4716-f6e8-4136-9a85-1279cb3ff4a2 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:48.263574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1423/2824] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:52:57.984073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:58.259274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=35 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:52:58.959040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070357e0-2642-4d20-9b4c-7761a1c46d49 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:59.025481 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070357e0-2642-4d20-9b4c-7761a1c46d49 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:59.030529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:59.031401 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:59.031683 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:59.031962 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] PATH_INFO: `/v3/users/211b9712181149088673e9ec04e82dc0` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:59.033849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=211b9712181149088673e9ec04e82dc0)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:59.035837 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:59.215533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '211b9712181149088673e9ec04e82dc0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:59.216323 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '211b9712181149088673e9ec04e82dc0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:59.230443 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '211b9712181149088673e9ec04e82dc0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:59.237272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '211b9712181149088673e9ec04e82dc0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:59.297719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '211b9712181149088673e9ec04e82dc0'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:52:59.305057 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1402/2825] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:52:58 2022] DELETE /identity/v3/users/211b9712181149088673e9ec04e82dc0 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:52:59.309773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b35d8b-5245-43a3-b8b9-c93adf1b9ee7 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:59.310274 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b35d8b-5245-43a3-b8b9-c93adf1b9ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:59.310392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b35d8b-5245-43a3-b8b9-c93adf1b9ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:59.327262 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-c3b35d8b-5245-43a3-b8b9-c93adf1b9ee7 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:52:59.328452 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c3b35d8b-5245-43a3-b8b9-c93adf1b9ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:59.407381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1424/2826] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:52:59.409298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:52:59.410896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:52:59.476439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe359d7-808b-40e2-9916-a5ff64add3eb admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:52:59.480316 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45f7ac-9958-45cf-86c4-12c7de8f8606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:52:59.480989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45f7ac-9958-45cf-86c4-12c7de8f8606 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:52:59.481327 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45f7ac-9958-45cf-86c4-12c7de8f8606 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:52:59.481605 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45f7ac-9958-45cf-86c4-12c7de8f8606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:52:59.491779 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45f7ac-9958-45cf-86c4-12c7de8f8606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:52:59.493263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45f7ac-9958-45cf-86c4-12c7de8f8606 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:52:59.503356 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1403/2827] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:52:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:53:00.008732 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b35d8b-5245-43a3-b8b9-c93adf1b9ee7 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:00.010208 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c3b35d8b-5245-43a3-b8b9-c93adf1b9ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:00.026668 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b35d8b-5245-43a3-b8b9-c93adf1b9ee7 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:00.032521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:00.033879 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:00.034200 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:00.034391 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] PATH_INFO: `/v3/projects/65f66da0127c4b08ac88f7eaabacf184` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:00.035726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f66da0127c4b08ac88f7eaabacf184)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:00.037635 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:00.201578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f66da0127c4b08ac88f7eaabacf184', 'request_id': 'req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:00.280362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f66da0127c4b08ac88f7eaabacf184', 'request_id': 'req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:00.290227 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f66da0127c4b08ac88f7eaabacf184', 'request_id': 'req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:00.300470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] The token cache is being invalidate because project 65f66da0127c4b08ac88f7eaabacf184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:53:00.300764 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:00.301450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:00.303066 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1425/2828] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:53:00 2022] DELETE /identity/v3/projects/65f66da0127c4b08ac88f7eaabacf184 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:53:00.307090 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45f7ac-9958-45cf-86c4-12c7de8f8606 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:00.316988 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45f7ac-9958-45cf-86c4-12c7de8f8606 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:00.335750 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:00.378466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:00.379346 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:00.379623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:00.379932 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] PATH_INFO: `/v3/users/23834a46141e4dbab69661ffbfbc6fc6` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:00.381115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] RBAC: Authorizing `identity:delete_user(user_id=23834a46141e4dbab69661ffbfbc6fc6)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:00.383093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:00.574667 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23834a46141e4dbab69661ffbfbc6fc6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:00.575223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23834a46141e4dbab69661ffbfbc6fc6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:00.587419 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23834a46141e4dbab69661ffbfbc6fc6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:00.594266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23834a46141e4dbab69661ffbfbc6fc6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:00.663078 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-34f1b9d6-1621-4578-8200-cd7d1709f06f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23834a46141e4dbab69661ffbfbc6fc6'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:00.670534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1404/2829] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:53:00 2022] DELETE /identity/v3/users/23834a46141e4dbab69661ffbfbc6fc6 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:53:01.104626 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:01.106139 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:01.125565 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21dfb6-dd3b-4a2c-80a7-4db71dc7186a admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:01.178993 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:01.179857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:01.180135 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:01.180427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] PATH_INFO: `/v3/projects/acbe617ee75546c4a430c206dae14aae` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:01.181617 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] RBAC: Authorizing `identity:delete_project(project_id=acbe617ee75546c4a430c206dae14aae)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:01.183525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:01.270620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:01.287334 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acbe617ee75546c4a430c206dae14aae', 'request_id': 'req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:01.348965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acbe617ee75546c4a430c206dae14aae', 'request_id': 'req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:01.355699 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acbe617ee75546c4a430c206dae14aae', 'request_id': 'req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:01.368065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] The token cache is being invalidate because project acbe617ee75546c4a430c206dae14aae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:53:01.368381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:01.369077 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-a36ee86f-aa97-41df-a6bc-5b8b1aa1056c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:01.371140 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1426/2830] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:53:01 2022] DELETE /identity/v3/projects/acbe617ee75546c4a430c206dae14aae => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:53:01.376423 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb17ae-aa18-43fe-bf5e-b796cbf6f9e3 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:01.377880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb17ae-aa18-43fe-bf5e-b796cbf6f9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:01.377880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb17ae-aa18-43fe-bf5e-b796cbf6f9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:01.403040 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-52eb17ae-aa18-43fe-bf5e-b796cbf6f9e3 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:01.405223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-52eb17ae-aa18-43fe-bf5e-b796cbf6f9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:01.476237 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1405/2831] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:53:01.485107 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c506-4f2b-4f3b-af8e-f8d4d3a48f57 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:01.485542 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c506-4f2b-4f3b-af8e-f8d4d3a48f57 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:01.487277 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c506-4f2b-4f3b-af8e-f8d4d3a48f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:01.507330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e9e6c506-4f2b-4f3b-af8e-f8d4d3a48f57 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:01.508656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e9e6c506-4f2b-4f3b-af8e-f8d4d3a48f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:01.517035 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1427/2832] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:53:01.522263 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb17ae-aa18-43fe-bf5e-b796cbf6f9e3 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:01.532673 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb17ae-aa18-43fe-bf5e-b796cbf6f9e3 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:01.538285 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ef977-87a0-4c4e-9cd8-040b1509da08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:01.540266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ef977-87a0-4c4e-9cd8-040b1509da08 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:01.540266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ef977-87a0-4c4e-9cd8-040b1509da08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:01.540266 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ef977-87a0-4c4e-9cd8-040b1509da08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:01.540615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ef977-87a0-4c4e-9cd8-040b1509da08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:53:01.540992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ef977-87a0-4c4e-9cd8-040b1509da08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:01.543008 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ef977-87a0-4c4e-9cd8-040b1509da08 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:01.556788 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1406/2833] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:53:01.561647 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6c506-4f2b-4f3b-af8e-f8d4d3a48f57 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:01.571192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6c506-4f2b-4f3b-af8e-f8d4d3a48f57 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:01.576076 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358db8f-d866-4f99-9c5a-27afadd85ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:01.576727 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358db8f-d866-4f99-9c5a-27afadd85ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:01.577024 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358db8f-d866-4f99-9c5a-27afadd85ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:01.577306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358db8f-d866-4f99-9c5a-27afadd85ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:01.578003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358db8f-d866-4f99-9c5a-27afadd85ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:01.580495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358db8f-d866-4f99-9c5a-27afadd85ba2 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:01.699410 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1428/2834] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:53:01 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:53:01.701148 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ef977-87a0-4c4e-9cd8-040b1509da08 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:01.712044 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ef977-87a0-4c4e-9cd8-040b1509da08 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:01.740574 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:01.773364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb14c92-efc5-4a69-9aad-7f5655ce8484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:01.774031 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb14c92-efc5-4a69-9aad-7f5655ce8484 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:01.774314 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb14c92-efc5-4a69-9aad-7f5655ce8484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:01.774587 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb14c92-efc5-4a69-9aad-7f5655ce8484 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:01.775284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb14c92-efc5-4a69-9aad-7f5655ce8484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:01.777265 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb14c92-efc5-4a69-9aad-7f5655ce8484 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:01.864035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1407/2835] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:53:01 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:53:01.868737 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358db8f-d866-4f99-9c5a-27afadd85ba2 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:01.879888 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358db8f-d866-4f99-9c5a-27afadd85ba2 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:01.884489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73812f53-f5dd-473b-87e2-98831873aa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:01.885147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73812f53-f5dd-473b-87e2-98831873aa2d admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:01.885454 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73812f53-f5dd-473b-87e2-98831873aa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:01.885726 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73812f53-f5dd-473b-87e2-98831873aa2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:01.886447 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73812f53-f5dd-473b-87e2-98831873aa2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:01.888006 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73812f53-f5dd-473b-87e2-98831873aa2d admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:01.899249 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1429/2836] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:53:01.903563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb14c92-efc5-4a69-9aad-7f5655ce8484 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:01.914780 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb14c92-efc5-4a69-9aad-7f5655ce8484 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:01.919714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e28cf-c2bd-480a-a8d8-29cc3ebd6309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:01.920259 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e28cf-c2bd-480a-a8d8-29cc3ebd6309 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:01.920542 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e28cf-c2bd-480a-a8d8-29cc3ebd6309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:01.920819 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e28cf-c2bd-480a-a8d8-29cc3ebd6309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:01.921589 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e28cf-c2bd-480a-a8d8-29cc3ebd6309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:01.923193 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e28cf-c2bd-480a-a8d8-29cc3ebd6309 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:01.933828 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1408/2837] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:53:01.938051 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73812f53-f5dd-473b-87e2-98831873aa2d admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:01.948948 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73812f53-f5dd-473b-87e2-98831873aa2d admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:01.953725 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a52128-ad4b-43d5-b8e9-c240a15d1738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:01.954553 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a52128-ad4b-43d5-b8e9-c240a15d1738 admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:01.954829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a52128-ad4b-43d5-b8e9-c240a15d1738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:01.955755 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a52128-ad4b-43d5-b8e9-c240a15d1738 admin admin] PATH_INFO: `/v3/projects/8877da2b84de4e0b8f1530218d03b18e/users/aae33f2968f64a128ad19c5e2f45d263/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:01.970523 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a52128-ad4b-43d5-b8e9-c240a15d1738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8877da2b84de4e0b8f1530218d03b18e, user_id=aae33f2968f64a128ad19c5e2f45d263, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:01.973193 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a52128-ad4b-43d5-b8e9-c240a15d1738 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:02.046773 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1430/2838] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:53:01 2022] PUT /identity/v3/projects/8877da2b84de4e0b8f1530218d03b18e/users/aae33f2968f64a128ad19c5e2f45d263/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:53:02.111919 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146286dc-969c-4254-a828-503baed1b9e3 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:02.112205 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146286dc-969c-4254-a828-503baed1b9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:02.112478 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146286dc-969c-4254-a828-503baed1b9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:02.139534 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-146286dc-969c-4254-a828-503baed1b9e3 None None] MFA Rules not processed for user `aae33f2968f64a128ad19c5e2f45d263`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:02.140737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-146286dc-969c-4254-a828-503baed1b9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:02.204455 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1409/2839] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:53:02.215521 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a52128-ad4b-43d5-b8e9-c240a15d1738 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:02.217317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c7a52128-ad4b-43d5-b8e9-c240a15d1738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:02.275835 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a52128-ad4b-43d5-b8e9-c240a15d1738 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:02.279434 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc6f6c-eb1a-4eea-a47f-60f99a62e450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:02.280079 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc6f6c-eb1a-4eea-a47f-60f99a62e450 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:02.280359 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc6f6c-eb1a-4eea-a47f-60f99a62e450 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:02.280633 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc6f6c-eb1a-4eea-a47f-60f99a62e450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:02.288750 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc6f6c-eb1a-4eea-a47f-60f99a62e450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:02.290244 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc6f6c-eb1a-4eea-a47f-60f99a62e450 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:02.299372 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1431/2840] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:53:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:53:12.201260 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:12.294922 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:23.091175 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146286dc-969c-4254-a828-503baed1b9e3 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:23.092619 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-146286dc-969c-4254-a828-503baed1b9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:23.159093 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146286dc-969c-4254-a828-503baed1b9e3 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:23.163649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:23.164512 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:23.164795 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:23.165071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] PATH_INFO: `/v3/users/134e3b513fc94678999784f92c6823e9` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:23.166294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=134e3b513fc94678999784f92c6823e9)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:23.168206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:23.369076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '134e3b513fc94678999784f92c6823e9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:23.372049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '134e3b513fc94678999784f92c6823e9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:23.386864 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '134e3b513fc94678999784f92c6823e9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:23.394338 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '134e3b513fc94678999784f92c6823e9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:23.485567 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '134e3b513fc94678999784f92c6823e9'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:23.492257 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1410/2841] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:53:23 2022] DELETE /identity/v3/users/134e3b513fc94678999784f92c6823e9 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:53:23.497399 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42e38-97c4-4fc5-b8a7-3c3b737839de None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:23.497915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42e38-97c4-4fc5-b8a7-3c3b737839de None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:23.498027 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42e38-97c4-4fc5-b8a7-3c3b737839de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:23.515742 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-fab42e38-97c4-4fc5-b8a7-3c3b737839de None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:23.516930 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fab42e38-97c4-4fc5-b8a7-3c3b737839de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:23.582392 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1432/2842] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:53:23.592870 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:23.594393 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:23.655529 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3e724-80f8-4fc4-aead-9778f2448e1e admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:23.658494 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1a32-9128-4d59-84fc-e642c2d6f06e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:23.659212 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1a32-9128-4d59-84fc-e642c2d6f06e service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:23.659495 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1a32-9128-4d59-84fc-e642c2d6f06e service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:23.659771 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1a32-9128-4d59-84fc-e642c2d6f06e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:23.671642 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf1a32-9128-4d59-84fc-e642c2d6f06e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:23.673032 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf1a32-9128-4d59-84fc-e642c2d6f06e service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:23.684171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1411/2843] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:53:24.048248 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab42e38-97c4-4fc5-b8a7-3c3b737839de None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:24.051237 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-fab42e38-97c4-4fc5-b8a7-3c3b737839de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:24.063309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab42e38-97c4-4fc5-b8a7-3c3b737839de None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:24.068254 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:24.069147 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:24.069517 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:24.069795 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] PATH_INFO: `/v3/projects/98f70aa6464146fd9c128deea9bf1c5b` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:24.071273 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f70aa6464146fd9c128deea9bf1c5b)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:24.073290 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:24.253587 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f70aa6464146fd9c128deea9bf1c5b', 'request_id': 'req-519f9e18-68a8-446d-9f96-2b189231b8c4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:24.359533 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f70aa6464146fd9c128deea9bf1c5b', 'request_id': 'req-519f9e18-68a8-446d-9f96-2b189231b8c4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:24.365595 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f70aa6464146fd9c128deea9bf1c5b', 'request_id': 'req-519f9e18-68a8-446d-9f96-2b189231b8c4'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:24.377011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] The token cache is being invalidate because project 98f70aa6464146fd9c128deea9bf1c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:53:24.377332 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:24.381190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-519f9e18-68a8-446d-9f96-2b189231b8c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:24.381190 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1433/2844] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:53:24 2022] DELETE /identity/v3/projects/98f70aa6464146fd9c128deea9bf1c5b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:53:26.181582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1a32-9128-4d59-84fc-e642c2d6f06e service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:26.182995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-8ddf1a32-9128-4d59-84fc-e642c2d6f06e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:26.196194 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1a32-9128-4d59-84fc-e642c2d6f06e service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:26.214115 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:26.252854 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:26.253743 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:26.254020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:26.254287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] PATH_INFO: `/v3/users/aae33f2968f64a128ad19c5e2f45d263` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:26.255569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae33f2968f64a128ad19c5e2f45d263)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:26.257977 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:26.396368 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae33f2968f64a128ad19c5e2f45d263'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:26.396958 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae33f2968f64a128ad19c5e2f45d263'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:26.409198 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae33f2968f64a128ad19c5e2f45d263'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:26.416503 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae33f2968f64a128ad19c5e2f45d263'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:26.496652 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae33f2968f64a128ad19c5e2f45d263'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:26.502856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1412/2845] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:53:26 2022] DELETE /identity/v3/users/aae33f2968f64a128ad19c5e2f45d263 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:53:26.507376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aa87b-3c51-41a9-bc96-bff8079dd80a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:26.508407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aa87b-3c51-41a9-bc96-bff8079dd80a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:26.508407 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aa87b-3c51-41a9-bc96-bff8079dd80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:26.527013 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-448aa87b-3c51-41a9-bc96-bff8079dd80a None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:26.528216 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-448aa87b-3c51-41a9-bc96-bff8079dd80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:26.586789 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1434/2846] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:53:26.596361 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:26.597820 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:26.654995 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71581d3b-fc3e-4bff-8fc2-8af884465b1d admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:26.658582 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b077-7a4d-4c49-95fe-9b01e2917704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:26.659224 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b077-7a4d-4c49-95fe-9b01e2917704 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:26.659510 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b077-7a4d-4c49-95fe-9b01e2917704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:26.659788 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b077-7a4d-4c49-95fe-9b01e2917704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:26.669627 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4b077-7a4d-4c49-95fe-9b01e2917704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:26.671073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4b077-7a4d-4c49-95fe-9b01e2917704 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:26.681114 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1413/2847] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:53:27.092676 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448aa87b-3c51-41a9-bc96-bff8079dd80a None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:27.099627 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-448aa87b-3c51-41a9-bc96-bff8079dd80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:27.107433 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448aa87b-3c51-41a9-bc96-bff8079dd80a None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:27.112134 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:27.113003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:27.113323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:27.113623 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] PATH_INFO: `/v3/projects/8877da2b84de4e0b8f1530218d03b18e` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:27.114811 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8877da2b84de4e0b8f1530218d03b18e)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:27.116730 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:27.139402 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:27.267995 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8877da2b84de4e0b8f1530218d03b18e', 'request_id': 'req-867b3061-a768-498f-bd99-7fa7d75a031c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:27.353915 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8877da2b84de4e0b8f1530218d03b18e', 'request_id': 'req-867b3061-a768-498f-bd99-7fa7d75a031c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:27.359838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8877da2b84de4e0b8f1530218d03b18e', 'request_id': 'req-867b3061-a768-498f-bd99-7fa7d75a031c'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:27.370798 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] The token cache is being invalidate because project 8877da2b84de4e0b8f1530218d03b18e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:53:27.371077 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:27.371704 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-867b3061-a768-498f-bd99-7fa7d75a031c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:53:27.373268 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1435/2848] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:53:27 2022] DELETE /identity/v3/projects/8877da2b84de4e0b8f1530218d03b18e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:53:27.378375 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f52521-9ba5-4a9e-b021-e01b9341d003 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:27.378908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f52521-9ba5-4a9e-b021-e01b9341d003 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:27.378908 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f52521-9ba5-4a9e-b021-e01b9341d003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:27.398355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-c4f52521-9ba5-4a9e-b021-e01b9341d003 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:27.400089 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-c4f52521-9ba5-4a9e-b021-e01b9341d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:27.442377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:27.462623 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1414/2849] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:53:27.470515 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ecaf5-b3d6-4607-9bad-646708b14be6 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:27.470806 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ecaf5-b3d6-4607-9bad-646708b14be6 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:27.471087 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ecaf5-b3d6-4607-9bad-646708b14be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:27.490785 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-e29ecaf5-b3d6-4607-9bad-646708b14be6 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:27.491996 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-e29ecaf5-b3d6-4607-9bad-646708b14be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:27.500224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1436/2850] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:53:27.504397 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f52521-9ba5-4a9e-b021-e01b9341d003 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:27.514048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f52521-9ba5-4a9e-b021-e01b9341d003 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:27.518994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506a346-e973-499c-8284-94ffcb7c4e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:27.519591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506a346-e973-499c-8284-94ffcb7c4e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:27.519873 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506a346-e973-499c-8284-94ffcb7c4e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:27.520191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506a346-e973-499c-8284-94ffcb7c4e92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:27.520994 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506a346-e973-499c-8284-94ffcb7c4e92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80434) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:53:27.521329 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506a346-e973-499c-8284-94ffcb7c4e92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:27.522921 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506a346-e973-499c-8284-94ffcb7c4e92 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:27.536048 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1415/2851] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:53:27.540607 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ecaf5-b3d6-4607-9bad-646708b14be6 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:27.550159 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ecaf5-b3d6-4607-9bad-646708b14be6 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:27.555033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2310b4d-c495-4d02-b4e7-9a92ad7364fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:27.555685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2310b4d-c495-4d02-b4e7-9a92ad7364fa admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:27.555965 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2310b4d-c495-4d02-b4e7-9a92ad7364fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:27.556265 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2310b4d-c495-4d02-b4e7-9a92ad7364fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:27.556975 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2310b4d-c495-4d02-b4e7-9a92ad7364fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:27.558958 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2310b4d-c495-4d02-b4e7-9a92ad7364fa admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:27.641404 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1437/2852] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:53:27 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:53:27.645826 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506a346-e973-499c-8284-94ffcb7c4e92 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:27.656273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506a346-e973-499c-8284-94ffcb7c4e92 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:27.712271 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251a12f-9168-40ff-9e94-efe0c305f0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:27.712933 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251a12f-9168-40ff-9e94-efe0c305f0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:27.713241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251a12f-9168-40ff-9e94-efe0c305f0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:27.713517 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251a12f-9168-40ff-9e94-efe0c305f0b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:27.714215 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251a12f-9168-40ff-9e94-efe0c305f0b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:27.716120 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251a12f-9168-40ff-9e94-efe0c305f0b9 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:27.828731 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1416/2853] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:53:27 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:53:27.832829 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2310b4d-c495-4d02-b4e7-9a92ad7364fa admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:27.843312 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2310b4d-c495-4d02-b4e7-9a92ad7364fa admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:27.847886 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e697e9-9db2-4b16-bd3e-391bcc0fb47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:27.848541 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e697e9-9db2-4b16-bd3e-391bcc0fb47b admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:27.848825 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e697e9-9db2-4b16-bd3e-391bcc0fb47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:27.849096 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e697e9-9db2-4b16-bd3e-391bcc0fb47b admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:27.849857 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e697e9-9db2-4b16-bd3e-391bcc0fb47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:27.851435 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e697e9-9db2-4b16-bd3e-391bcc0fb47b admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:27.862263 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1438/2854] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:53:27.871152 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251a12f-9168-40ff-9e94-efe0c305f0b9 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:27.881296 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251a12f-9168-40ff-9e94-efe0c305f0b9 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:27.885942 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7b35a-b0f8-49b8-b9c1-baa28a832ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:27.886604 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7b35a-b0f8-49b8-b9c1-baa28a832ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:27.886916 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7b35a-b0f8-49b8-b9c1-baa28a832ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:27.887191 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7b35a-b0f8-49b8-b9c1-baa28a832ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:27.887923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7b35a-b0f8-49b8-b9c1-baa28a832ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:27.889566 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7b35a-b0f8-49b8-b9c1-baa28a832ce1 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:27.899929 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1417/2855] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:53:27.904369 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e697e9-9db2-4b16-bd3e-391bcc0fb47b admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:27.914696 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e697e9-9db2-4b16-bd3e-391bcc0fb47b admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:27.919233 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7dda1-8de9-4c3a-8093-1fd6676ea9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:27.920099 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7dda1-8de9-4c3a-8093-1fd6676ea9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:27.920376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7dda1-8de9-4c3a-8093-1fd6676ea9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:27.920653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7dda1-8de9-4c3a-8093-1fd6676ea9ad admin admin] PATH_INFO: `/v3/projects/e9905cd8264640fc99e83d869129d85a/users/485159da9ec14f44b398c3cc618d3d84/roles/df5932e731294f7498a6983a730628bd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:27.935387 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7dda1-8de9-4c3a-8093-1fd6676ea9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9905cd8264640fc99e83d869129d85a, user_id=485159da9ec14f44b398c3cc618d3d84, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:27.938022 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7dda1-8de9-4c3a-8093-1fd6676ea9ad admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:28.000145 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1439/2856] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:53:27 2022] PUT /identity/v3/projects/e9905cd8264640fc99e83d869129d85a/users/485159da9ec14f44b398c3cc618d3d84/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:53:28.006226 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e45330-c7a4-4032-8098-65bb887a7d8b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:28.006520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e45330-c7a4-4032-8098-65bb887a7d8b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:28.006796 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e45330-c7a4-4032-8098-65bb887a7d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:28.024730 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-e1e45330-c7a4-4032-8098-65bb887a7d8b None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:28.025996 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-e1e45330-c7a4-4032-8098-65bb887a7d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:28.090206 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1418/2857] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:53:28.099716 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7dda1-8de9-4c3a-8093-1fd6676ea9ad admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:28.101489 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f7e7dda1-8de9-4c3a-8093-1fd6676ea9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:28.158849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7dda1-8de9-4c3a-8093-1fd6676ea9ad admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:28.162525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7fb6f6-d731-4d8c-9730-0ba3968ef84a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:28.163166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fb6f6-d731-4d8c-9730-0ba3968ef84a service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:28.163450 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fb6f6-d731-4d8c-9730-0ba3968ef84a service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:28.163728 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fb6f6-d731-4d8c-9730-0ba3968ef84a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:28.173417 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7fb6f6-d731-4d8c-9730-0ba3968ef84a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:28.174850 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7fb6f6-d731-4d8c-9730-0ba3968ef84a service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:28.184453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1440/2858] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:53:29.259490 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33808a-fff4-4b34-ad02-ba9891d0a99b None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:29.260081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33808a-fff4-4b34-ad02-ba9891d0a99b None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:29.260081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33808a-fff4-4b34-ad02-ba9891d0a99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:29.287047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-cc33808a-fff4-4b34-ad02-ba9891d0a99b None None] MFA Rules not processed for user `485159da9ec14f44b398c3cc618d3d84`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:29.291020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-cc33808a-fff4-4b34-ad02-ba9891d0a99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:29.330158 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:29.361860 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1419/2859] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:53:29.370157 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7fb6f6-d731-4d8c-9730-0ba3968ef84a service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:29.371549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3b7fb6f6-d731-4d8c-9730-0ba3968ef84a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:29.426374 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:29.432250 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7fb6f6-d731-4d8c-9730-0ba3968ef84a service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:29.435887 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216b5c2-0138-475c-9d2a-79a8ebe5cc59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:29.439362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b5c2-0138-475c-9d2a-79a8ebe5cc59 service cinder] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:29.439362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b5c2-0138-475c-9d2a-79a8ebe5cc59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:29.439362 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b5c2-0138-475c-9d2a-79a8ebe5cc59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:29.445462 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216b5c2-0138-475c-9d2a-79a8ebe5cc59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:29.446946 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216b5c2-0138-475c-9d2a-79a8ebe5cc59 service cinder] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:29.455934 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1441/2860] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:53:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:53:31.116569 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddd7d3-a5a8-4db7-9c55-fb9216319fa6 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:31.117112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddd7d3-a5a8-4db7-9c55-fb9216319fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:31.117112 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddd7d3-a5a8-4db7-9c55-fb9216319fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:31.135609 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-afddd7d3-a5a8-4db7-9c55-fb9216319fa6 None None] MFA Rules not processed for user `485159da9ec14f44b398c3cc618d3d84`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:31.136807 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-afddd7d3-a5a8-4db7-9c55-fb9216319fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:31.144030 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1420/2861] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:53:31.374565 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216b5c2-0138-475c-9d2a-79a8ebe5cc59 service cinder] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:53:31.376002 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-c216b5c2-0138-475c-9d2a-79a8ebe5cc59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:31.388507 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216b5c2-0138-475c-9d2a-79a8ebe5cc59 service cinder] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:53:31.392059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df427a3b-b9ab-43b8-961e-59cecd0c9f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:53:31.392705 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df427a3b-b9ab-43b8-961e-59cecd0c9f97 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:31.392988 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df427a3b-b9ab-43b8-961e-59cecd0c9f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:31.393309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df427a3b-b9ab-43b8-961e-59cecd0c9f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:31.395337 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-df427a3b-b9ab-43b8-961e-59cecd0c9f97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:31.405279 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df427a3b-b9ab-43b8-961e-59cecd0c9f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:53:31.406692 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df427a3b-b9ab-43b8-961e-59cecd0c9f97 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:53:31.414735 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1442/2862] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:53:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:53:41.129965 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=12 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:41.411203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=11 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:53:50.104221 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e40f2d-1952-4b00-9e21-268c8fdd3e93 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:53:50.105383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e40f2d-1952-4b00-9e21-268c8fdd3e93 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:53:50.105383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e40f2d-1952-4b00-9e21-268c8fdd3e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:53:50.122902 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-90e40f2d-1952-4b00-9e21-268c8fdd3e93 None None] MFA Rules not processed for user `485159da9ec14f44b398c3cc618d3d84`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:53:50.124085 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-90e40f2d-1952-4b00-9e21-268c8fdd3e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:53:50.130882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1421/2863] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:53:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:54:00.117842 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:54:13.085638 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c7c17-5c53-4ae1-9fb4-f8f14e5f713a None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:13.086119 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c7c17-5c53-4ae1-9fb4-f8f14e5f713a None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:13.086224 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c7c17-5c53-4ae1-9fb4-f8f14e5f713a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:13.114033 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-2a7c7c17-5c53-4ae1-9fb4-f8f14e5f713a None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:54:13.115236 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2a7c7c17-5c53-4ae1-9fb4-f8f14e5f713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:13.177422 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1443/2864] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jul 14 11:54:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:54:13.357751 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e40f2d-1952-4b00-9e21-268c8fdd3e93 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:13.359273 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-90e40f2d-1952-4b00-9e21-268c8fdd3e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:13.430223 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e40f2d-1952-4b00-9e21-268c8fdd3e93 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:13.435520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ea5f7-8b8a-40fb-ae17-276042e7382e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:13.436172 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ea5f7-8b8a-40fb-ae17-276042e7382e service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:13.436456 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ea5f7-8b8a-40fb-ae17-276042e7382e service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:13.436735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ea5f7-8b8a-40fb-ae17-276042e7382e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:13.445701 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ea5f7-8b8a-40fb-ae17-276042e7382e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:13.447285 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ea5f7-8b8a-40fb-ae17-276042e7382e service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:13.455404 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1422/2865] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:54:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:54:23.171065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=32 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:54:23.452174 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=32 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:54:31.004926 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95cde2-7bef-4a2e-9f23-fe6d1c00f9ec None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:31.005444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95cde2-7bef-4a2e-9f23-fe6d1c00f9ec None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:31.005549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95cde2-7bef-4a2e-9f23-fe6d1c00f9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:31.054604 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-9e95cde2-7bef-4a2e-9f23-fe6d1c00f9ec None None] MFA Rules not processed for user `0bc624eba860448eab54b516bd344a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:54:31.055695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-9e95cde2-7bef-4a2e-9f23-fe6d1c00f9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:31.072307 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1444/2866] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jul 14 11:54:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:54:31.084282 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ea5f7-8b8a-40fb-ae17-276042e7382e service swift] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:31.104704 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ea5f7-8b8a-40fb-ae17-276042e7382e service swift] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:31.109738 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6885ad-8979-402f-a641-afed009a9edb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a5ec793f014cefb9d0a0d7258bbb59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:31.111010 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6885ad-8979-402f-a641-afed009a9edb service swift] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:31.111287 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6885ad-8979-402f-a641-afed009a9edb service swift] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:31.111559 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6885ad-8979-402f-a641-afed009a9edb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:31.119886 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6885ad-8979-402f-a641-afed009a9edb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:31.121538 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6885ad-8979-402f-a641-afed009a9edb service swift] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:31.129793 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1423/2867] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jul 14 11:54:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:54:34.000299 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95cde2-7bef-4a2e-9f23-fe6d1c00f9ec None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:34.014830 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95cde2-7bef-4a2e-9f23-fe6d1c00f9ec None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:34.019309 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:34.020192 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:34.020535 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:34.020815 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] PATH_INFO: `/v3/users/485159da9ec14f44b398c3cc618d3d84` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:34.022059 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=485159da9ec14f44b398c3cc618d3d84)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:34.023982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:34.211166 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485159da9ec14f44b398c3cc618d3d84'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:54:34.217852 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485159da9ec14f44b398c3cc618d3d84'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:54:34.230443 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485159da9ec14f44b398c3cc618d3d84'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:54:34.306701 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485159da9ec14f44b398c3cc618d3d84'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:54:34.307377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-07d4c2e4-6f1f-4fa8-9d0e-431f1fa23650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485159da9ec14f44b398c3cc618d3d84'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:54:34.314370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1445/2868] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:54:33 2022] DELETE /identity/v3/users/485159da9ec14f44b398c3cc618d3d84 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:54:34.572243 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6885ad-8979-402f-a641-afed009a9edb service swift] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:34.584355 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6885ad-8979-402f-a641-afed009a9edb service swift] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:34.642520 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:34.643389 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:34.643668 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:34.643939 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] PATH_INFO: `/v3/projects/e9905cd8264640fc99e83d869129d85a` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:34.645183 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9905cd8264640fc99e83d869129d85a)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:34.647117 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:34.787876 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9905cd8264640fc99e83d869129d85a', 'request_id': 'req-07fda568-037a-4f2b-ac5a-8fc89a2491ec'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:54:34.794100 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9905cd8264640fc99e83d869129d85a', 'request_id': 'req-07fda568-037a-4f2b-ac5a-8fc89a2491ec'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:54:34.799649 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9905cd8264640fc99e83d869129d85a', 'request_id': 'req-07fda568-037a-4f2b-ac5a-8fc89a2491ec'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:54:34.896968 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] The token cache is being invalidate because project e9905cd8264640fc99e83d869129d85a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:54:34.897229 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:54:34.897829 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-07fda568-037a-4f2b-ac5a-8fc89a2491ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:54:34.899398 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1424/2869] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:54:34 2022] DELETE /identity/v3/projects/e9905cd8264640fc99e83d869129d85a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:54:34.904345 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548df927-4dec-4d0d-8e2e-ed68b0c267ae None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:34.905306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548df927-4dec-4d0d-8e2e-ed68b0c267ae None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:34.905306 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548df927-4dec-4d0d-8e2e-ed68b0c267ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:34.923352 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-548df927-4dec-4d0d-8e2e-ed68b0c267ae None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:54:34.924581 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-548df927-4dec-4d0d-8e2e-ed68b0c267ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:34.986127 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1446/2870] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:54:34.993895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb2981-5dfd-4448-9370-a016360b3051 None None] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:34.994348 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb2981-5dfd-4448-9370-a016360b3051 None None] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:34.994613 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb2981-5dfd-4448-9370-a016360b3051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:35.014150 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.auth.core [None req-a1eb2981-5dfd-4448-9370-a016360b3051 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80434) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:54:35.015358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-a1eb2981-5dfd-4448-9370-a016360b3051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:35.023654 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1425/2871] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:54:35.027936 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548df927-4dec-4d0d-8e2e-ed68b0c267ae None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:35.038406 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548df927-4dec-4d0d-8e2e-ed68b0c267ae None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:35.043100 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddce76c-e979-49fc-86d8-00fc19254210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:35.043784 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce76c-e979-49fc-86d8-00fc19254210 admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:35.044064 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce76c-e979-49fc-86d8-00fc19254210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:35.044338 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce76c-e979-49fc-86d8-00fc19254210 admin admin] PATH_INFO: `/v3/domains` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:35.045063 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddce76c-e979-49fc-86d8-00fc19254210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:54:35.045556 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddce76c-e979-49fc-86d8-00fc19254210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:35.047168 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddce76c-e979-49fc-86d8-00fc19254210 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:35.059261 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1447/2872] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jul 14 11:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:54:35.064289 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb2981-5dfd-4448-9370-a016360b3051 None None] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:35.073555 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb2981-5dfd-4448-9370-a016360b3051 None None] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:35.078167 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404e14d-a488-47ed-9849-2d58940850c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:35.078849 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404e14d-a488-47ed-9849-2d58940850c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:35.079162 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404e14d-a488-47ed-9849-2d58940850c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:35.079433 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404e14d-a488-47ed-9849-2d58940850c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:35.080113 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404e14d-a488-47ed-9849-2d58940850c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:35.082038 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404e14d-a488-47ed-9849-2d58940850c3 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:35.178431 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1426/2873] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jul 14 11:54:35 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:54:35.182598 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddce76c-e979-49fc-86d8-00fc19254210 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:35.193379 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddce76c-e979-49fc-86d8-00fc19254210 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:35.210069 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:54:35.248421 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef025327-2b0c-4eab-adc0-6decf1d2efc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:35.249409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef025327-2b0c-4eab-adc0-6decf1d2efc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:35.249856 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef025327-2b0c-4eab-adc0-6decf1d2efc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:35.250380 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef025327-2b0c-4eab-adc0-6decf1d2efc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:35.251893 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef025327-2b0c-4eab-adc0-6decf1d2efc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:35.254289 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef025327-2b0c-4eab-adc0-6decf1d2efc8 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:35.366298 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1448/2874] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jul 14 11:54:35 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:54:35.370184 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404e14d-a488-47ed-9849-2d58940850c3 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:35.381053 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404e14d-a488-47ed-9849-2d58940850c3 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:35.385769 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98457c-2eb3-47e1-9a2d-7fcf31ad4f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:35.386466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98457c-2eb3-47e1-9a2d-7fcf31ad4f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:35.386745 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98457c-2eb3-47e1-9a2d-7fcf31ad4f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:35.387020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98457c-2eb3-47e1-9a2d-7fcf31ad4f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:35.387752 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98457c-2eb3-47e1-9a2d-7fcf31ad4f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:35.389381 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98457c-2eb3-47e1-9a2d-7fcf31ad4f0c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:35.400195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1427/2875] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:54:35.403983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef025327-2b0c-4eab-adc0-6decf1d2efc8 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:35.414606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef025327-2b0c-4eab-adc0-6decf1d2efc8 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:35.419228 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2da29-0e79-40ad-afbb-c51e6cacebac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:35.419891 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2da29-0e79-40ad-afbb-c51e6cacebac admin admin] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:35.420178 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2da29-0e79-40ad-afbb-c51e6cacebac admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:35.420456 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2da29-0e79-40ad-afbb-c51e6cacebac admin admin] PATH_INFO: `/v3/roles` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:35.421292 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2da29-0e79-40ad-afbb-c51e6cacebac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:35.423436 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2da29-0e79-40ad-afbb-c51e6cacebac admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:35.433882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1449/2876] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jul 14 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:54:35.438037 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98457c-2eb3-47e1-9a2d-7fcf31ad4f0c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:35.447930 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98457c-2eb3-47e1-9a2d-7fcf31ad4f0c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:35.452528 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b9fbd-9370-48e1-aad6-16f096eebee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:35.453432 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b9fbd-9370-48e1-aad6-16f096eebee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:35.453818 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b9fbd-9370-48e1-aad6-16f096eebee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:35.454099 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b9fbd-9370-48e1-aad6-16f096eebee5 admin admin] PATH_INFO: `/v3/projects/81884daddee04f478f047f2c309ad433/users/d282558617e24bcdb1bf6bddbab7cddd/roles/df5932e731294f7498a6983a730628bd` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:35.468524 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b9fbd-9370-48e1-aad6-16f096eebee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81884daddee04f478f047f2c309ad433, user_id=d282558617e24bcdb1bf6bddbab7cddd, role_id=df5932e731294f7498a6983a730628bd)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:35.471892 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b9fbd-9370-48e1-aad6-16f096eebee5 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:35.475081 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:54:35.604615 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1428/2877] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jul 14 11:54:35 2022] PUT /identity/v3/projects/81884daddee04f478f047f2c309ad433/users/d282558617e24bcdb1bf6bddbab7cddd/roles/df5932e731294f7498a6983a730628bd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:54:35.609781 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f252ab-e380-4cbc-8627-816578a8a767 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:35.610729 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f252ab-e380-4cbc-8627-816578a8a767 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:35.611008 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f252ab-e380-4cbc-8627-816578a8a767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:35.629205 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-18f252ab-e380-4cbc-8627-816578a8a767 None None] MFA Rules not processed for user `e0d5b9edc140499bb612222f6572ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:54:35.633625 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-18f252ab-e380-4cbc-8627-816578a8a767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:35.687700 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1450/2878] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:54:35.697086 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b9fbd-9370-48e1-aad6-16f096eebee5 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:35.698869 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0a6b9fbd-9370-48e1-aad6-16f096eebee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:35.755466 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b9fbd-9370-48e1-aad6-16f096eebee5 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:35.759241 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8de97-9ed1-496c-a54e-9b8325adb1b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:35.759884 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8de97-9ed1-496c-a54e-9b8325adb1b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:35.760171 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8de97-9ed1-496c-a54e-9b8325adb1b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:35.760450 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8de97-9ed1-496c-a54e-9b8325adb1b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:35.770900 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8de97-9ed1-496c-a54e-9b8325adb1b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:35.772377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8de97-9ed1-496c-a54e-9b8325adb1b7 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:35.782722 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1429/2879] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:54:36.980606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2b384-580d-4187-b1d1-6f6fb47635e9 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:36.981174 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2b384-580d-4187-b1d1-6f6fb47635e9 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:36.981283 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2b384-580d-4187-b1d1-6f6fb47635e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:37.008357 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-60f2b384-580d-4187-b1d1-6f6fb47635e9 None None] MFA Rules not processed for user `d282558617e24bcdb1bf6bddbab7cddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:54:37.009606 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-60f2b384-580d-4187-b1d1-6f6fb47635e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:37.079470 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1451/2880] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:54:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:54:37.090573 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8de97-9ed1-496c-a54e-9b8325adb1b7 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:37.092270 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ffe8de97-9ed1-496c-a54e-9b8325adb1b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:37.155591 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8de97-9ed1-496c-a54e-9b8325adb1b7 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:37.159858 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbf4f8-48cb-4710-8ee5-14184de7d41b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:37.160509 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbf4f8-48cb-4710-8ee5-14184de7d41b service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:37.160882 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbf4f8-48cb-4710-8ee5-14184de7d41b service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:37.161071 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbf4f8-48cb-4710-8ee5-14184de7d41b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:37.170016 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbf4f8-48cb-4710-8ee5-14184de7d41b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:37.171461 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbf4f8-48cb-4710-8ee5-14184de7d41b service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:37.181272 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1430/2881] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:54:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:54:38.915041 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b493d4-78a1-437f-b6bd-0af8159917ba None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:38.915571 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b493d4-78a1-437f-b6bd-0af8159917ba None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:38.915691 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b493d4-78a1-437f-b6bd-0af8159917ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:38.933112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-f6b493d4-78a1-437f-b6bd-0af8159917ba None None] MFA Rules not processed for user `d282558617e24bcdb1bf6bddbab7cddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:54:38.934382 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f6b493d4-78a1-437f-b6bd-0af8159917ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:38.941457 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1452/2882] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:54:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:54:38.977922 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbf4f8-48cb-4710-8ee5-14184de7d41b service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:38.979311 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-27cbf4f8-48cb-4710-8ee5-14184de7d41b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:39.065982 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbf4f8-48cb-4710-8ee5-14184de7d41b service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:39.069706 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8327b-52c6-44fe-b63d-f5ed81c0d046 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1a7213cbf6448b7ba6570a52a4c7c0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:39.070351 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8327b-52c6-44fe-b63d-f5ed81c0d046 service glance] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:39.070632 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8327b-52c6-44fe-b63d-f5ed81c0d046 service glance] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:39.079027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8327b-52c6-44fe-b63d-f5ed81c0d046 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:39.079027 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5ec8327b-52c6-44fe-b63d-f5ed81c0d046 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:39.109252 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=100 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:54:39.154254 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8327b-52c6-44fe-b63d-f5ed81c0d046 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:39.155725 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8327b-52c6-44fe-b63d-f5ed81c0d046 service glance] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:39.175945 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1431/2883] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:54:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:54:39.836339 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b493d4-78a1-437f-b6bd-0af8159917ba None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:39.837845 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-f6b493d4-78a1-437f-b6bd-0af8159917ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:39.843856 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8327b-52c6-44fe-b63d-f5ed81c0d046 service glance] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:39.854161 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8327b-52c6-44fe-b63d-f5ed81c0d046 service glance] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:39.857946 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a92800-f5f2-4dcc-89be-1554ad9d01c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:39.858669 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a92800-f5f2-4dcc-89be-1554ad9d01c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:39.859075 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a92800-f5f2-4dcc-89be-1554ad9d01c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:39.859358 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a92800-f5f2-4dcc-89be-1554ad9d01c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:39.861841 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-30a92800-f5f2-4dcc-89be-1554ad9d01c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:39.865323 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:54:39.881622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a92800-f5f2-4dcc-89be-1554ad9d01c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:39.883073 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a92800-f5f2-4dcc-89be-1554ad9d01c3 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:39.895177 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1432/2884] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:54:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:54:39.910026 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b493d4-78a1-437f-b6bd-0af8159917ba None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:39.914992 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215da39-160d-48c3-bc43-be26226bd56a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:39.915766 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215da39-160d-48c3-bc43-be26226bd56a service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:39.916329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215da39-160d-48c3-bc43-be26226bd56a service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:39.916329 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215da39-160d-48c3-bc43-be26226bd56a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:39.918048 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-2215da39-160d-48c3-bc43-be26226bd56a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:39.930782 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215da39-160d-48c3-bc43-be26226bd56a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:39.932359 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215da39-160d-48c3-bc43-be26226bd56a service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:39.942833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1453/2885] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:54:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:54:49.395344 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a92800-f5f2-4dcc-89be-1554ad9d01c3 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:49.399284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-30a92800-f5f2-4dcc-89be-1554ad9d01c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:49.413049 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a92800-f5f2-4dcc-89be-1554ad9d01c3 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:49.419129 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0bf8f-9ddb-4c51-b2b4-baadfcaac159 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:49.419838 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0bf8f-9ddb-4c51-b2b4-baadfcaac159 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:49.420207 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0bf8f-9ddb-4c51-b2b4-baadfcaac159 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:49.420546 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0bf8f-9ddb-4c51-b2b4-baadfcaac159 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:49.422636 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-29c0bf8f-9ddb-4c51-b2b4-baadfcaac159 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:49.436469 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0bf8f-9ddb-4c51-b2b4-baadfcaac159 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:49.438234 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0bf8f-9ddb-4c51-b2b4-baadfcaac159 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:49.449799 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1433/2886] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:54:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:54:49.958658 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=22 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:54:57.670985 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12409992-385e-4ba3-a920-dd7c27000d3e None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:57.671663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12409992-385e-4ba3-a920-dd7c27000d3e None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:57.671663 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12409992-385e-4ba3-a920-dd7c27000d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:57.688709 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-12409992-385e-4ba3-a920-dd7c27000d3e None None] MFA Rules not processed for user `1a71cad9c9114c5cb0bf8b4fe982e4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:54:57.689983 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-12409992-385e-4ba3-a920-dd7c27000d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:57.746258 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1454/2887] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:54:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:54:58.066575 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0bf8f-9ddb-4c51-b2b4-baadfcaac159 service nova] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:54:58.076334 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0bf8f-9ddb-4c51-b2b4-baadfcaac159 service nova] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:54:58.079997 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cc051-8aae-4ad8-8b70-cab245718eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:54:58.080638 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cc051-8aae-4ad8-8b70-cab245718eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:54:58.080920 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cc051-8aae-4ad8-8b70-cab245718eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:54:58.081228 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cc051-8aae-4ad8-8b70-cab245718eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:54:58.082981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ac4cc051-8aae-4ad8-8b70-cab245718eb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:54:58.093670 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4cc051-8aae-4ad8-8b70-cab245718eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:54:58.095094 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4cc051-8aae-4ad8-8b70-cab245718eb0 service neutron] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:54:58.104549 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1434/2888] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:54:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:55:01.853409 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12409992-385e-4ba3-a920-dd7c27000d3e None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:55:01.863849 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12409992-385e-4ba3-a920-dd7c27000d3e None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:55:01.869005 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d47d4-4495-454f-8bb2-23fa449f7952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:55:01.869744 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d47d4-4495-454f-8bb2-23fa449f7952 service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:55:01.870032 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d47d4-4495-454f-8bb2-23fa449f7952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:55:01.870319 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d47d4-4495-454f-8bb2-23fa449f7952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:55:01.872315 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-607d47d4-4495-454f-8bb2-23fa449f7952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:55:01.883953 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d47d4-4495-454f-8bb2-23fa449f7952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:55:01.885982 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d47d4-4495-454f-8bb2-23fa449f7952 service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:55:01.895900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1455/2889] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:55:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:55:02.760195 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cc051-8aae-4ad8-8b70-cab245718eb0 service neutron] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:55:02.761735 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-ac4cc051-8aae-4ad8-8b70-cab245718eb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:55:02.785600 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cc051-8aae-4ad8-8b70-cab245718eb0 service neutron] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:55:02.790438 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2faf8-aefe-43da-95fd-8e11a9d69940 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:55:02.791211 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2faf8-aefe-43da-95fd-8e11a9d69940 service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:55:02.791737 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2faf8-aefe-43da-95fd-8e11a9d69940 service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:55:02.792139 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2faf8-aefe-43da-95fd-8e11a9d69940 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:55:02.795020 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-faa2faf8-aefe-43da-95fd-8e11a9d69940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:55:02.816237 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2faf8-aefe-43da-95fd-8e11a9d69940 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:55:02.818035 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2faf8-aefe-43da-95fd-8e11a9d69940 service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:55:02.830571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1435/2890] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:55:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:55:11.891736 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=31 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:55:12.824574 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=50 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:55:47.046525 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dea13-24da-4df6-91f8-5042f993f5ea None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:55:47.048940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dea13-24da-4df6-91f8-5042f993f5ea None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:55:47.048940 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dea13-24da-4df6-91f8-5042f993f5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:55:47.065618 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-a33dea13-24da-4df6-91f8-5042f993f5ea None None] MFA Rules not processed for user `d282558617e24bcdb1bf6bddbab7cddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:55:47.066862 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-a33dea13-24da-4df6-91f8-5042f993f5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:55:47.074314 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1456/2891] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:55:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:55:57.059968 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:56:52.014890 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2faf8-aefe-43da-95fd-8e11a9d69940 service nova] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:56:52.021417 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-faa2faf8-aefe-43da-95fd-8e11a9d69940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:56:52.031439 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2faf8-aefe-43da-95fd-8e11a9d69940 service nova] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:56:52.036496 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6437bd-2d11-41c0-9561-10652afed1e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:56:52.037141 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6437bd-2d11-41c0-9561-10652afed1e5 service placement] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:56:52.037487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6437bd-2d11-41c0-9561-10652afed1e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:56:52.037742 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6437bd-2d11-41c0-9561-10652afed1e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:56:52.039583 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-7d6437bd-2d11-41c0-9561-10652afed1e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:56:52.052714 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6437bd-2d11-41c0-9561-10652afed1e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:56:52.054342 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6437bd-2d11-41c0-9561-10652afed1e5 service placement] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:56:52.064992 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1436/2892] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:56:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:56:52.143141 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dea13-24da-4df6-91f8-5042f993f5ea None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:56:52.154112 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dea13-24da-4df6-91f8-5042f993f5ea None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:56:52.158882 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:56:52.159900 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:56:52.160181 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:56:52.160453 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] PATH_INFO: `/v3/users/d282558617e24bcdb1bf6bddbab7cddd` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:56:52.161695 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d282558617e24bcdb1bf6bddbab7cddd)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:56:52.163605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:56:52.338652 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd282558617e24bcdb1bf6bddbab7cddd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:56:52.345475 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd282558617e24bcdb1bf6bddbab7cddd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:56:52.358102 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd282558617e24bcdb1bf6bddbab7cddd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:56:52.438967 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd282558617e24bcdb1bf6bddbab7cddd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:56:52.439605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd282558617e24bcdb1bf6bddbab7cddd'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:56:52.446680 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1457/2893] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:56:52 2022] DELETE /identity/v3/users/d282558617e24bcdb1bf6bddbab7cddd => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:56:52.674880 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6437bd-2d11-41c0-9561-10652afed1e5 service placement] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:56:52.686047 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6437bd-2d11-41c0-9561-10652afed1e5 service placement] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:56:52.743931 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:56:52.744784 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:56:52.745066 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:56:52.745383 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] PATH_INFO: `/v3/projects/81884daddee04f478f047f2c309ad433` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:56:52.746540 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81884daddee04f478f047f2c309ad433)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:56:52.748504 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:56:52.862145 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81884daddee04f478f047f2c309ad433', 'request_id': 'req-908e983c-8139-4f4c-bac3-8f16156f6d98'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:56:52.869418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81884daddee04f478f047f2c309ad433', 'request_id': 'req-908e983c-8139-4f4c-bac3-8f16156f6d98'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:56:52.875418 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81884daddee04f478f047f2c309ad433', 'request_id': 'req-908e983c-8139-4f4c-bac3-8f16156f6d98'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:56:52.952585 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] The token cache is being invalidate because project 81884daddee04f478f047f2c309ad433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:56:52.952787 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:56:52.953446 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:56:52.954989 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1437/2894] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:56:52 2022] DELETE /identity/v3/projects/81884daddee04f478f047f2c309ad433 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:57:02.443023 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:57:02.949187 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:57:04.212300 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:57:04.213796 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:57:04.284065 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff163d-c342-4f9a-bc36-ef3803fb5c90 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:57:04.343377 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c9e55-6d1b-40ea-b519-e47c5dba4947 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:57:04.344025 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c9e55-6d1b-40ea-b519-e47c5dba4947 service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:57:04.344302 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c9e55-6d1b-40ea-b519-e47c5dba4947 service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:57:04.344574 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c9e55-6d1b-40ea-b519-e47c5dba4947 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:57:04.346656 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8c5c9e55-6d1b-40ea-b519-e47c5dba4947 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:57:04.359317 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c9e55-6d1b-40ea-b519-e47c5dba4947 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:57:04.360892 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c9e55-6d1b-40ea-b519-e47c5dba4947 service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:57:04.371297 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1458/2895] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:57:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:57:07.921592 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:57:07.923060 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:57:07.982391 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e983c-8139-4f4c-bac3-8f16156f6d98 admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:57:07.986092 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824ece1-4c7b-4efe-a739-3aa667e0c464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f12e348368ac42c696623b4a8f781977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:57:07.989339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824ece1-4c7b-4efe-a739-3aa667e0c464 service cinder] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:57:07.989339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824ece1-4c7b-4efe-a739-3aa667e0c464 service cinder] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:57:07.989339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824ece1-4c7b-4efe-a739-3aa667e0c464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:57:07.989339 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0824ece1-4c7b-4efe-a739-3aa667e0c464 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:57:08.048026 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824ece1-4c7b-4efe-a739-3aa667e0c464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:57:08.049533 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824ece1-4c7b-4efe-a739-3aa667e0c464 service cinder] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:57:08.059076 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1438/2896] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:57:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:57:14.366675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=58 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:57:16.411885 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00af00-aff5-4819-8540-bdb689919967 None None] REQUEST_METHOD: `POST` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:57:16.412549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00af00-aff5-4819-8540-bdb689919967 None None] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:57:16.412549 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00af00-aff5-4819-8540-bdb689919967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:57:16.431239 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.auth.core [None req-dd00af00-aff5-4819-8540-bdb689919967 None None] MFA Rules not processed for user `1a71cad9c9114c5cb0bf8b4fe982e4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:57:16.432448 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dd00af00-aff5-4819-8540-bdb689919967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:57:16.439414 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1459/2897] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jul 14 11:57:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:57:18.054364 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=48 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:57:26.425833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=3 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:57:37.320584 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824ece1-4c7b-4efe-a739-3aa667e0c464 service cinder] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:57:37.322284 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-0824ece1-4c7b-4efe-a739-3aa667e0c464 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:57:37.400294 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824ece1-4c7b-4efe-a739-3aa667e0c464 service cinder] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:57:37.406630 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa9d24-de05-4a96-a17a-6a646e0bc79f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147d764ef17042dbb7ea179c29711588', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:57:37.407340 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa9d24-de05-4a96-a17a-6a646e0bc79f service nova] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:57:37.407622 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa9d24-de05-4a96-a17a-6a646e0bc79f service nova] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:57:37.407896 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa9d24-de05-4a96-a17a-6a646e0bc79f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:57:37.409895 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f1fa9d24-de05-4a96-a17a-6a646e0bc79f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:57:37.425719 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa9d24-de05-4a96-a17a-6a646e0bc79f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:57:37.427101 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa9d24-de05-4a96-a17a-6a646e0bc79f service nova] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:57:37.439416 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1439/2898] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:57:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:57:47.434055 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=34 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:59:23.286203 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00af00-aff5-4819-8540-bdb689919967 None None] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:59:23.287590 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-dd00af00-aff5-4819-8540-bdb689919967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:59:23.343838 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00af00-aff5-4819-8540-bdb689919967 None None] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:59:23.347376 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3ea21-6901-477e-a432-357d1c7b1bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481ef56b1cfa465a8efc59736a34e412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:59:23.348016 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3ea21-6901-477e-a432-357d1c7b1bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:59:23.348294 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3ea21-6901-477e-a432-357d1c7b1bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:59:23.348564 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3ea21-6901-477e-a432-357d1c7b1bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:59:23.350342 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-66e3ea21-6901-477e-a432-357d1c7b1bfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:59:23.416675 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3ea21-6901-477e-a432-357d1c7b1bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:59:23.418072 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3ea21-6901-477e-a432-357d1c7b1bfc service neutron] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:59:23.428444 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1460/2899] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:59:26.910377 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa9d24-de05-4a96-a17a-6a646e0bc79f service nova] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:59:26.912219 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-f1fa9d24-de05-4a96-a17a-6a646e0bc79f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:59:26.927186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa9d24-de05-4a96-a17a-6a646e0bc79f service nova] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:59:26.932563 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53210b-d7e8-48cb-96f4-f00a891208a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:59:26.933290 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53210b-d7e8-48cb-96f4-f00a891208a2 service placement] REQUEST_METHOD: `GET` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:59:26.933580 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53210b-d7e8-48cb-96f4-f00a891208a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:59:26.933859 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53210b-d7e8-48cb-96f4-f00a891208a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:59:26.936915 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5c53210b-d7e8-48cb-96f4-f00a891208a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:59:26.949202 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53210b-d7e8-48cb-96f4-f00a891208a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:59:26.950804 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53210b-d7e8-48cb-96f4-f00a891208a2 service placement] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:59:26.961328 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1440/2900] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 11:59:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:59:30.481909 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3ea21-6901-477e-a432-357d1c7b1bfc service neutron] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:59:30.483403 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-66e3ea21-6901-477e-a432-357d1c7b1bfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:59:30.495809 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3ea21-6901-477e-a432-357d1c7b1bfc service neutron] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:59:30.501045 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:59:30.502047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:59:30.502330 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:59:30.502605 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] PATH_INFO: `/v3/users/1a71cad9c9114c5cb0bf8b4fe982e4d5` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:59:30.503801 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a71cad9c9114c5cb0bf8b4fe982e4d5)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:59:30.505765 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:59:30.654068 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a71cad9c9114c5cb0bf8b4fe982e4d5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:30.661071 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a71cad9c9114c5cb0bf8b4fe982e4d5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:30.673555 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a71cad9c9114c5cb0bf8b4fe982e4d5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:30.725653 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a71cad9c9114c5cb0bf8b4fe982e4d5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:30.726195 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a71cad9c9114c5cb0bf8b4fe982e4d5'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:30.732499 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1461/2901] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:59:30 2022] DELETE /identity/v3/users/1a71cad9c9114c5cb0bf8b4fe982e4d5 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:59:30.939042 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53210b-d7e8-48cb-96f4-f00a891208a2 service placement] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:59:30.951875 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53210b-d7e8-48cb-96f4-f00a891208a2 service placement] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:59:31.010014 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:59:31.010926 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:59:31.011209 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:59:31.011483 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] PATH_INFO: `/v3/projects/7119e46a06b54c2984b21f32d9c156e3` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:59:31.012678 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7119e46a06b54c2984b21f32d9c156e3)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:59:31.014618 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:59:31.132304 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7119e46a06b54c2984b21f32d9c156e3', 'request_id': 'req-5db70e01-0e47-499c-a184-ce6824c2b24c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:31.138777 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7119e46a06b54c2984b21f32d9c156e3', 'request_id': 'req-5db70e01-0e47-499c-a184-ce6824c2b24c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:31.144341 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7119e46a06b54c2984b21f32d9c156e3', 'request_id': 'req-5db70e01-0e47-499c-a184-ce6824c2b24c'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:31.215776 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] The token cache is being invalidate because project 7119e46a06b54c2984b21f32d9c156e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:59:31.216064 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:31.216705 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:31.218778 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1441/2902] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:59:30 2022] DELETE /identity/v3/projects/7119e46a06b54c2984b21f32d9c156e3 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:59:31.222770 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:59:31.224287 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:59:31.240003 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beb6af-f13c-4497-9e79-5e3f294ff649 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:59:31.258713 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:59:31.298685 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:59:31.299901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:59:31.299901 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:59:31.300136 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] PATH_INFO: `/v3/users/d2f8dbd621a74ea29ff3621a49a5de32` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:59:31.301495 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f8dbd621a74ea29ff3621a49a5de32)` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:59:31.303393 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:59:31.483047 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f8dbd621a74ea29ff3621a49a5de32'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:31.490944 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f8dbd621a74ea29ff3621a49a5de32'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:31.516831 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f8dbd621a74ea29ff3621a49a5de32'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:31.582899 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f8dbd621a74ea29ff3621a49a5de32'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:31.583833 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.notifications [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f8dbd621a74ea29ff3621a49a5de32'} {{(pid=80433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:31.593269 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1462/2903] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jul 14 11:59:31 2022] DELETE /identity/v3/users/d2f8dbd621a74ea29ff3621a49a5de32 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:59:31.846726 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] Authenticating user token {{(pid=80434) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:59:31.848218 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.fernet_utils [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80434) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:59:31.863981 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db70e01-0e47-499c-a184-ce6824c2b24c admin admin] Validating token access rules against request {{(pid=80434) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:59:31.922395 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d5b9edc140499bb612222f6572ae20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da372d98ddc34559ac820df9de6d2798', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80434) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:59:31.923277 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:59:31.923557 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:59:31.923830 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] PATH_INFO: `/v3/projects/bc8c54170e7b45399e6ff7218f53ab8e` {{(pid=80434) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:59:31.924999 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8c54170e7b45399e6ff7218f53ab8e)` {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:59:31.926923 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] RBAC: Authorization granted {{(pid=80434) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:59:31.947949 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:59:32.028295 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8c54170e7b45399e6ff7218f53ab8e', 'request_id': 'req-58c53fbe-9750-40b4-a311-cf395a25af5e'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:32.034648 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8c54170e7b45399e6ff7218f53ab8e', 'request_id': 'req-58c53fbe-9750-40b4-a311-cf395a25af5e'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:32.040571 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8c54170e7b45399e6ff7218f53ab8e', 'request_id': 'req-58c53fbe-9750-40b4-a311-cf395a25af5e'} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:32.130905 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] The token cache is being invalidate because project bc8c54170e7b45399e6ff7218f53ab8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80434) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:59:32.131186 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:32.131741 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG keystone.notifications [None req-58c53fbe-9750-40b4-a311-cf395a25af5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80434) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:59:32.133298 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: [pid: 80434|app: 0|req: 1442/2904] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jul 14 11:59:31 2022] DELETE /identity/v3/projects/bc8c54170e7b45399e6ff7218f53ab8e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:59:41.586620 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:59:42.205487 dv-cinder-849803-1-iscsi devstack@keystone.service[80434]: DEBUG dbcounter [-] [80434] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80434) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 12:02:10.214578 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] Authenticating user token {{(pid=80433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 12:02:10.216011 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 12:02:10.356370 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755fb02-c917-4ff4-a333-0a02d1f24b64 admin admin] Validating token access rules against request {{(pid=80433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 12:02:10.414866 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10aa01-6ca2-4e4a-be6b-37dec8e28283 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa879d581c6d44e18a8703e66ffbd549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69cec1c5d7884e1b94e44e7b586ef19a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 12:02:10.415511 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10aa01-6ca2-4e4a-be6b-37dec8e28283 service placement] REQUEST_METHOD: `GET` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 12:02:10.415820 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10aa01-6ca2-4e4a-be6b-37dec8e28283 service placement] SCRIPT_NAME: `/identity` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 12:02:10.416586 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10aa01-6ca2-4e4a-be6b-37dec8e28283 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 12:02:10.418394 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.fernet_utils [None req-3a10aa01-6ca2-4e4a-be6b-37dec8e28283 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 12:02:10.429427 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10aa01-6ca2-4e4a-be6b-37dec8e28283 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 12:02:10.430996 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10aa01-6ca2-4e4a-be6b-37dec8e28283 service placement] RBAC: Authorization granted {{(pid=80433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 12:02:10.441381 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: [pid: 80433|app: 0|req: 1463/2905] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jul 14 12:02:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 12:02:20.437177 dv-cinder-849803-1-iscsi devstack@keystone.service[80433]: DEBUG dbcounter [-] [80433] Writing DB stats keystone:SELECT=89 {{(pid=80433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}